Toyota and Generative AI: It’s Here, and This is How We’re Using It

The news about Artificial intelligence (AI) can sound alarming. With the latest hype, it sounds like robots could soon be gaining sentience and humans could slowly be replaced by thinking computers.

That isn’t reality, however. AI is already augmenting human skills and capabilities in many products and services that we use every day, and it has the potential to automate many tasks. The goal is a future where humans can spend more time focusing on creative, strategic and enjoyable activities.

Toyota has been using AI for years, within the enterprise as well as in the products we offer. We’re building our capabilities to meet the next generation of challenges that anticipate customers’ needs and exceed their expectations.

Going Strong Since 2016

Toyota’s initial goal in 2016 was to engineer a resilient cloud safety system, and that led to the development of Safety Connect, a service powered by Drivelink from software company Toyota Connected North America (TCNA). The Safety Connect service is designed to leverage key data points from the vehicle to identify when a collision has occurred and send an automatic notification to call center agents. Should the driver become unconscious, telematics information can provide a more complete picture of the situation, enabling agents to contact authorities faster when it’s needed most.

In parallel and through ingenuity, TCNA engineers built proofs of concept to see how they could enable AI-driven services as well. As such, the “Hey Toyota” and “Hey Lexus” Virtual Agents were born to support voice interactions in the vehicle. Launched in 2021, vehicles with the latest Toyota Audio Multimedia or Lexus Interface multimedia system have been using AI via Virtual Agent voice commands for audio, climate control, and other functions – even the ability to tell you a joke if you’re needing a pick-me-up.

From your SOS button to a number of other services, Safety Connect, powered by Toyota Connected’s Drivelink telematics platform, enables a number of safety and convenience technologies in Toyota and Lexus vehicles.

Since then, Toyota has been steadily building, testing, and deploying AI-powered features to enhance the safety, quality, and customer experience with our products.

Minor bumps, scratches and dings can have significant impact to a vehicle’s residual value. Existing in-vehicle technology was not deigned for what we consider as micro-collisions, or any type of collision that falls below the standard collision-detection thresholds. Leveraging ideas across the organization, Toyota teams have built a robust machine learning pipeline that is designed to predict low-speed collisions and differentiate them from harsh braking and cornering events. We are actively taking our models to the next level by working to classify the micro-collisions by type, body location and severity to assess our drivers and passengers’ well being and if the vehicle may need repair under the surface more so than what can be visibly detected.

Vehicle maintenance has also been a focus of AI-driven enhancements. Connected vehicles have hundreds of sensors, and we have been using data from these vehicles to build machine learning models for the most common maintenance items, including batteries, brakes, tires, and oil, and are currently investigating dozens of other components, using daily streaming data from millions of connected and consented vehicles. This suite of predictive maintenance models will help make customers aware of potential maintenance needs prior to component failures, so they can enjoy more reliable mobility experiences.

Toyota Connected engineers are looking into Predictive Maintenance concepts, which are designed to use AI algorithms to detect when parts may need to be replaced before the driver feels or hears changes in performance, such as with brake pads and rotors.

“The auto industry is in a state of revolution rather than evolution,” said Toyota Motor North America (TMNA) President and CEO Ted Ogawa. “AI is helping accelerate what we offer our customers, transforming Toyota into the mobility company we need to be to compete in this changing landscape. While Safety Connect and our latest infotainment systems are among the first of many AI-enabled technologies we use, it is just the beginning as our companies leverage the best people and technologies to help provide our drivers and passengers with intuitive, delightful experiences.”

Ogawa added, “We have made significant investments in AI talent at TCNA, and we are excited to leverage this great talent to lead Toyota companies in this space.”

The Latest Features, Now in your Toyota and Lexus

Destination Assist, which is also powered by Drivelink, is a powerful tool for drivers who enjoy their navigation systems but may need help with the inputs. Traditionally, when activated through an active Drive Connect trial or subscription, drivers would speak with a call center agent who would transmit directions back into their cars.

Starting in May 2023, Drivelink began rolling out an automated version of Destination Assist on select models, which leverages AI, freeing agents to help customers with more critical or complex needs.

Since then, the average Destination Assist call time has dropped from 102 to 62 seconds, and quality stands at a 92% completion rate – with the other 8% routed to a live agent.

“AI is complementing what Toyota companies do well – helping get customers on the road safely and confidently,” said TCNA Chief Technology Officer Brian Kursar. “We’ve been building AI services into our core competencies for years now and look to be a center of excellence to help the services we offer to be faster and stronger for all of our customers – both within Toyota and to the millions of Toyota and Lexus drivers on the road today.”

The Future: Generative AI and Large Language Models

Toyota has been quickly gaining expertise in Generative AI, which leverages vast databases and large language models to learn, grow and enhance how engineers enhance their core competencies and provide quicker, more intelligent products and services.

In April 2023, TCNA collaborated with Lexus marketing to launch a Generative AI vehicle art activation at the New York International Auto Show, where guests had the opportunity to create a picture of the 2023 Lexus RX or RZ on their own terms. Want to see how our vehicles look on your favorite beach, on a mountainous vacation destination, or even on a futuristic Mars landscape? It’s now just a few key swipes away.

Toyota is bringing the owner’s manual back down to Earth, as well. Wonder what the icon with squiggly lines is on your dashboard? In the near future, our Generative AI products will be able to tell you in a matter of seconds while on the go, rather than sifting through 600+ pages of a paper manual at a stop, saving time and materials needed to produce the manual. Over millions of vehicles, that will make a significant impact to our sustainability efforts while also reducing costs.

Finally, Generative AI unlocks intelligent in-vehicle experiences for our customers using expanded personal voice agent features. For instance, Generative AI can cut down on time searching far and wide throughout the internet to find roadside destinations, or it can even enable in-vehicle, contextual-based games. Think of “I Spy,” but leveraging the internet, Generative AI and navigation coordinates instead.

This year, TMNA formed a centralized Enterprise AI group, partnering with TCNA to focus on continued research and experimentation of new AI technologies, especially Generative AI. The group is helping coordinate efforts between internal divisions and across Toyota affiliates, regions, and group companies to make these use cases a reality. In addition, the Enterprise AI group co-leads a steering board to ensure that Responsible AI practices are implemented across the enterprise.

AI: The Path Forward at Toyota

AI is unlocking a world of possibilities, and Toyota companies are already well on their way to using its promise to unify legacy coding in networks with newer systems, lower costs, boost productivity, and create better products. Customers will soon see and feel the benefits in their cars, trucks, and SUVs.

“Toyota has the potential to be at the forefront of this seismic technology shift,” said Kursar. “Our early exploration and successes show AI is here to stay. We will invest in the frameworks to use AI responsibly so we can use it as a force multiplier to accelerate the transformation of our business into an electrified mobility company.”