The Next-Gen Mercedes-Benz Operating System Has An AI Virtual Assistant

Mercedes-Benz is getting ready to introduce its newest operating system, succinctly called MB.OS, after years of development. Designed in-house, MB.OS made its public debut at CES 2024 with a unique AI virtual assistant, a variety of audio and gaming partner apps, and a three-dimensional gaming engine for more complex graphics.

Artificially Flavored
The MB.OS software, which is distinct from the MBUX infotainment system, is quicker and more efficient than current Mercedes systems, helping improve responsiveness and processing power. One of the most obvious upgrades from old to new is the MBUX Virtual Assistant, which makes use of generative AI and a proprietary large language model to provide more natural speech feedback and conversations, detailed suggestions based on routines, and moods that will echo the driver’s.

As on today’s MBUX voice assistant – better known as Hey Mercedes – the Virtual Assistant can respond to natural language prompts like “I’m hungry,” for example. However, where Hey Mercedes would simply recommend a few nearby restaurants and call it a day, the MBUX Virtual Assistant will summarize the menu and reviews to help the driver decide on a destination. The AI system will also suggest functions and routes based on the driver’s routine, including reserving parking in advance or rescheduling calendar events if you’re running late.

Virtual Assistant even tailors its functionality to the driver’s state of mind. If it senses frustration, its prompts and responses will be shorter and more direct. If you’re in a good mood, Virtual Assistant’s voice will be cheerier and more conversational. It can even portray emotions via its “face,” which is actually a star field on the center display. The field will change colors and icons will appear to convey “like,” “celebrate,” “laugh,” and “confirm.”

Where Am I?
3D navigation will show up on cars equipped with MB.OS, making use of the gaming engine for sharp graphics and detailed driving directions. The navigation display will incorporate driver assistance information, including surrounding vehicles and road signs, to build trust between the safety systems and the person behind the wheel. The detailed display will also recommend specific lanes and display augmented-reality instructions in real time.

The new 3D navigation also shows surroundings, including buildings, time of day, and weather conditions. Mercedes says this will help drivers make better decisions and see where they’re headed, reducing confusion in tight urban confines.

Boombox Bookworm
A new version of MBUX will show up on cars equipped with MB.OS, and the infotainment system will now include built-in Audible and Amazon Music. When playing books, radio dramas, and music that are Dolby Atmos–capable, the audio system will synchronize the speakers, ambient lighting, and in-seat exciters for a more immersive experience.

The new MBUX will also feature a partnership with Sony Pictures Entertainment for streaming video and Antstream for arcade-style video games. The former will include major titles like Gran Turismo and Spider Man: Into The Spider-Verse, while the latter has a variety of retro arcade games, including a running game developed with Mercedes itself. Predictably, those functions will only work when the car is parked or making use of Level 3 driver assistance.

Dealership-Bound
The first application of MB.OS will appear on the production version of the Mercedes-Benz Concept CLA-Class, due sometime this year. Built on the so-called MMA platform, the next-generation CLA will be optimized for an EV, but will also offer ICE powertrains.

MB.OS will spread to other Mercedes product lines as they’re updated and redesigned, although some of the system’s functionality will be available via over-the-air updates as part of the MBUX infotainment package.