Meet the QNX concept team: Alex James, software engineer

We continue our spotlight on the QNX concept development team with Alex James, who gives us his impressions of the Bentley and the buzz at 2013 CES.

Besides attending CES, Alex worked on the latest QNX technology concept car from conception to launch — an amazing experience for any software engineer.

Working with bleeding-edge technologies sometimes brings unexpected challenges, along with interesting opportunities, as Alex discovered.

If you haven't had a chance to meet the other team members, you can read their stories here.



The challenge of creating an (auto)mobile user experience

On March 12, I had the honor of joining a distinguished group of panelists at a luncheon for the Los Angeles Motor Press Guild. The panelists included:


The purpose of the panel was to share information on trends in the connected car space and in the automotive application ecosystem. The panel was well attended, with journalists from publications like the New York Times, and with representatives from companies like Alpine, Beats by Dr. Dre, Hyundai, and Toyota.

Two things stood out for me. First, the press really picked up on the need for solutions that can offer ease of use, upgradeability, and reliability while also reducing distraction and liability. Second, an expert witness hired by car companies to testify in Lemon Law suits told the panel that he was already being hired to provide testimony in cases involving in-vehicle electronics. He speculated that the technology described on the panel was going to “make him rich.”

His comments help illustrate a point. A car isn’t a mobile phone. OEMs and end-users may want the same kind of fresh and updateable experience that a phone can provide, but unlike a phone, an in-car infotainment system must be simple to use even while you’re driving down the highway. Such systems offer the ideal environment for a hard real-time OS that can also enable the latest consumer technologies and applications in a reliable and easy-to-use way.

Jim Pisz mentioned a sign he saw at the Geneva Motor Show. The sign said “Don’t Worry, Be Appy.” That sign makes me realize that the industry is at a crossroads. OEMs want access to consumer app developers and, in some cases, the apps themselves. At the same time they want a reliable solution that they won’t have to “worry” about. With QNX’s pedigree of reliability and amazing app ecosystem, we are uniquely positioned to help OEMs build “appy” cars, without the worry.

Traveling on reserve power

Or how a new kind of electro-mobility can be fun. A guest post from Thomas Fleischmann of Elektrobit Automotive.

Thomas Fleishmann
Imagine you are always driving on reserve power. You find this difficult? Get used to it — and welcome to the new era of electro-mobility! Vehicles like the Chevy Volt, with its secondary combustion engine, are already addressing this challenge — but having to support two types of driving technology can be challenging and costly.

So how can electro-mobility, which is supported by software, be implemented meaningfully? And how do we get drivers to accept it? Certainly not by abandoning the driver with nothing but a nicely animated display of the car’s battery condition.

Let's assume you live in a big city. You don’t even own a car. Instead, you subscribe to a certain number of hours of travel time with your favorite car brand. In addition to your S or XL subscription for four weeks a year, you get access to a fossil-fuel engine for your vacation in another state or country twice a year.

In the morning you find and reserve a car with your BlackBerry phone and get into it at a nearby charge-point. The HMI adapts to your profile settings automatically — your friends, contacts, addresses, and music are already there. The navigation system is your energy consultant; it tells you, based on traffic conditions and topography, how far you can drive with this car and, at the appropriate time, suggests an available electric socket within easy reach. Or the system warns you to turn back soon if you want to arrive home safely. After parking the car successfully, your smartphone guides you for the last few kilometers by bus or subway to your destination — it knows the way and easily adopts the data of your navigation system.

Using software solutions like EB GUIDE or EB street director and frameworks like the QNX CAR application platform you can concentrate on creating the end-user experience and transform the journey on reserve power into something fun and convenient — suddenly an electric vehicle becomes a smart mobility concept.

Check this out; I fried it last week. The yellow center represents the area in which you can drive and go back home. The white area represents the range you can drive, depending on traffic or topography:





Thomas Fleischmann is Senior Product Manager at Elektrobit Automotive responsible for the HMI solution EB GUIDE. Contact him at Thomas.Fleischmann@elektrobit.com.


Fortune Profiles QNX Software Systems

In the latest issue of Fortune magazine, Kurt Wagner takes a look at the history of QNX and specifically highlights our leadership in the automotive industry. In the article, Kurt mentions the automotive companies we are working with — including Audi, Toyota, BMW, Porsche, Honda and more — and also provides a look at the technology concept car we unveiled at CES 2013, based on a Bentley Continental GTC.

To quote Kurt, “…the $191,000 luxury vehicle quickly became a must-see attraction, not for its curvaceous sheet metal but for its futuristic dashboard inside, featuring a massive 17-inch touchscreen, 3-D maps, even videoconferencing.”

You can read the full article in the April 8 issue of Fortune or on the web here (requires a subscription). Want to see more of this gorgeous technology concept car? Go behind the scenes and take a full tour.



The isolation imperative: protecting software components in an ISO 26262 system

Software components can be impolite, if not downright delinquent. For instance, a component might:

  • rob other components of CPU time
  • rob other components of file descriptors and other system resources
  • access the private memory of other components
  • corrupt data shared with other components
  • create a deadlock or livelock situation with other components

Shameful, I know. But in all seriousness, this sort of behavior can wreak havoc in a safety-critical system. For instance, let's say that a component starts to perform a CPU-intensive calculation just as the system enters a failure condition. Will that component hog the CPU and prevent an alarm process from running?

The answer, of course, is that it damn well better not.

It becomes important, then, to prevent components from interfering with one another. In fact, this principle is baked into the ISO 26262 functional safety standard for road vehicles, which defines interference as:

    "...the presence of cascading failures from a sub-element with no ASIL [Automotive Safety Integrity Level] assigned, or a lower ASIL assigned, to a sub-element with a higher ASIL assigned leading to the violation of a safety requirement of the element”

To put it crudely, less important stuff can't stop more important stuff from happening.

So how do you prevent interference? One approach is through isolation. For instance, a system may implement spatial isolation between application processes. This would include mechanisms for interprocess communication and interprocess locking that prevent one process from inadvertently affecting another.

Mind you, there are multiple types of interference, so you need to implement multiple forms, or axes, of isolation. Time for a picture:




In general, you need to determine what does, and what doesn't, need to be isolated. You also need to identify which components are apt to be delinquent and build a cage around them to protect more critical components. Which brings me to a recent paper by my inestimable colleagues Chris Hobbs and Yi Zheng. It's titled "Protecting Software Components from Interference in an ISO 26262 System," and it explores techniques that can help you:

  • implement the component isolation required by ISO 26262
  • demonstrate that such isolation has been implemented

And while you're at it, check out the other titles in our "safe" whitepaper series. These include "The Dangers of Over-Engineering a Safe System" and "Ten Truths about Building Safe Embedded Software Systems."

And don't worry: there's nothing delinquent about downloading all of them.

Hello, Bentley: Using Sensory speech technology to create a natural user experience

By Bernard Brafman, Sensory Inc., and Justin Moon, QNX Software Systems

In-vehicle infotainment systems are becoming more and more complex, and integral to the overall driving experience. As this trend continues, it will become increasingly important to create systems that support multiple forms of user interaction. If you’re driving, the last thing you want to do is enter a destination manually, or search for your favorite artist in a playlist by using a touch screen, jog wheel, or other manual input method. Drivers want and require a user experience that is both simple and natural; integration of speech recognition technology goes a long way toward achieving that goal.

In fact, speech recognition is a key component of the latest QNX technology concept car, a modified Bentley Continental GT. The speech rec system lets you plot a route or select your favorite artist using natural speech, but it goes even further by letting you simply ask the car to perform an action. Leveraging Sensory’s FluentSoft SDK, more specifically the TrulyHandsfreeTM Voice Control technology, the QNX concept development team implemented keyword spotting techniques to interact with the vehicle.

So how does this work? Well, let’s say you’re in Vegas and need directions to the Wynn Casino. To engage the cloud-based Watson speech system, you simply say “Hello Bentley” — no need to push a button. You then complete the request by saying “Take me to the Wynn Casino.” FluentSoft, along with the architecture of the advanced speech recognition system included in the QNX CAR platform, allowed the team to create this seamless, easily implemented, and well-executed voice interaction experience.


When you say “Hello Bentley,” the QNX concept car displays a visual prompt at the top of the screen, indicating that the speech rec system is now listening for natural speech or directed commands.

Multiple triggers
The team plans to further utilize Sensory technology in future concept car releases. The current implementation has the single “Hello Bentley” trigger, which engages the speech system. But TrulyHandsfree Voice Control supports multiple active triggers as well as a robust recognition vocabulary to create a rich command-and-control user experience that doesn’t require prompts or pauses. Thus, it’s possible to create a hybrid system that is seamless and transparent to the user. For instance, “Hello Bentley air 68 degrees” or “Hello Bentley what time is it in Tokyo?” can both be executed flawlessly, regardless of which speech rec system is engaged.

A matter of choice
For an even more personalized experience, this technology can allow drivers to create their own custom trigger with a simple one-time enrollment process that verifies their identity as a voice password or identifies one of several previously enrolled. This creates a custom experience not only by letting you choose your own trigger phrases (come on now, who hasn’t named their car at some point?), but also by recalling individual preferences such as seat position, steering wheel position, and multimedia presets.

Look for these enhanced features in concept cars to come!



Bernard Brafman is vice president of business development for Sensory, Inc., responsible for strategic business partnerships. He received his MSEE from Stanford University. Contact Bernard at bbrafman@sensoryinc.com

Justin Moon is a global technical evangelist for the automotive business development team at QNX Software Systems.