“Hey, Toyota,” you say, taking a moment to ponder what you’d like, “I fancy some coffee.” And, just like that, your 2023 bZ4X electric vehicle obliges, bringing up a list of nearby cafes, their ratings and asking you if you’d like the route to one for your mid-day break.

Seems easy, but behind the scenes, there’s a complex web of commands, hand-offs, and processing working its way through your car and the cloud to make sure your wish is the car’s command.

Ryan Oehler, product owner at Toyota Connected, an independent software and innovation company, leads the team responsible for Toyota’s new Intelligent Assistant, available in both the Toyota Audio Multimedia and Lexus Interface for the North American market. The new infotainment system is available in 2022 model year vehicles like the Toyota Tundra and Lexus NX. Connected Services trials are included on these vehicles, including Drive Connect, which offers Intelligent Assistant, Cloud Navigation and Destination Assist.

Oehler said with the Intelligent Assistant, which is cloud-based and supplements the onboard Voice Assistant, there has been a steady ramp up to meet both mainstream American customers and tech-savvy “power users” where they’re at with intuitive features for today’s drivers and a slew of more advanced features in the pipeline.

It all starts with automatic speech recognition (ASR). When the driver pushes the talk button in the vehicle, it transcribes that speech to text both in the vehicle’s embedded computer and over the cloud.

“It’s doing things like looking at the waveforms of the audio, and it’s actually able to understand, phonetically, what that translates to and formulate transcriptions based on that audio input,” Oehler said.

The technology is able to recognize commands, parsing through accents, dialects and even different pitches, frequencies and tones to recognize what to do next. Because of the markets where it is currently sold – the U.S., Canada and Mexico – it’s even designed to be able to recognize commands in Spanish and French in addition to English.

The transcribed command text is then processed by Toyota Connected’s machine learning models in the cloud – what is the intent of the words being used, from audio commands to windshield wipers or even finding a five-star-rated burger restaurant nearby. From there, it’s fed back to the vehicle where notes are compared between the embedded and cloud voice assistants to ensure the car is most accurately executing the command the user actually wants.

“This is where the Toyota Connected magic comes in,” Oehler said.

First up, the computer determines top-line intent – temperature, audio, navigation, etc. Then comes sub-intent – specific temperature, a song or that ice cream shop that always seems to hit the spot. If it’s a music request for a new J-Pop song, and you’re already listening to your favorite streaming platform, the head unit will attempt to find the song on that platform. If it’s not there, it will switch to the next best option, such as satellite radio.

The system is constantly getting smarter, looking at accuracy and tweaking commands to add more ways to ask for specific functions. The new infotainment technologies that have brought forth the Intelligent Assistant made their way to the Lexus NX, LX, and Toyota Tundra for 2022 but are already being introduced in more models for 2023 in North America. As it proliferates to more models, the speed of the machine learning system’s ability to, well, learn will only accelerate.

Of course, if a customer doesn’t want to use the Intelligent Assistant, they can use the embedded Voice Assistant as well for more basic functions.

“Certain audiences expect to interact in completely different ways,” Oehler said. “Younger audiences that are more familiar with their modern voice assistants tend to operate in a more fluid manner, whereas people that are less familiar would say, ‘Navigation’ and then ‘Texas’ in a series of steps,” for instance. To help the user, we’ve added more contextual things, such as prompts, to help guide them through this new user experience. One of the other important things that we’re learning is what our customers want to use on a daily basis.”

And it does so with the quality experience customers have come to expect from Toyota.

More features are planned, as well as bolstering the features already available. Ask the Intelligent Assistant to tell you a joke. Believe us when we say its wry wit is worth a trial or subscription.*

The sense of humor only recently joined the Virtual Assistant’s repertoire as the team of engineers observed customers asking for daily funnies.

“There’s a lot of opportunity,” Oehler said. “There’s value with a voice assistant because it adds depth to the experience, is smart, and can and likely will strengthen consumer confidence as more people use it and evolves in future iterations.”