Haptic Surface for Intelligent Vehicle Human-Machine Interface

2024

Tianyu Yu

Advisor: Haipeng Mi

Colleborators: Kejin Yu, Zengrui Li, Shiji Dai, Zhihao Yao, Xuezhu Wang, Yao Lu, Yijie Guo, Guanhong Liu

Sponsors: Tsinghua University & Beijing Mercedes-Benz

Intelligent vehicles have been evolving with new functions and user experiences. However, in-vehicle Human–Machine Interfaces (HMIs), as the primary medium for effective information interaction, remain limited in their feedback modalities, leading to issues such as increased attentional load due to complex operations or insufficient consideration of users’ emotional states. In contrast, haptic surfaces, enabled by emerging smart material technologies, offer an alternative approach for interactions that are intuitive, physically embedded, and emotionally aware.

In this project, we explore the design of haptic surfaces for in-vehicle HMIs based on smart material technologies. We present a concrete design case to illustrate this concept: a steering wheel with pneumatic haptic feedback that provides assistance during driving, such as information cues and emotional regulation. We designed and fabricated a functional prototype and conducted preliminary user studies to evaluate its effectiveness. In addition, we present a set of conceptual renderings to further extend the idea of haptic surfaces for in-vehicle HMIs. Finally, this work was exhibited in the 2024 postgraduate exhibition of the Academy of Art & Design, Tsinghua University.

Background

Evolving Human-Machine Interface (HMI) for Intelligent Vehicle

Intelligent vehicles have entered the public eye. In order to support the new functions and experiences for users, the in-vehicle human-machine interface (HMI) is also constantly evolving. New functions, such as information services and autonomous driving, have been recently incorporated, with the in-car interfaces gradually evolving for touchscreen interaction, head-up displays (HUD), and human–machine collaborative driving. Meanwhile, as intelligent vehicles are increasingly envisioned as a “third space” for users’ social, entertainment, and emotional experiences, in-car interfaces have also incorporated novel multimodal interfaces—such as virtual reality interfaces or social robotic interfaces—to enhance the playfulness and immersiveness of user experiences.

Haptic Feedback for In-vehicle HMI

However, existing in-vehicle HMIs are primarily delivered through visual and auditory (voice-based) channels, which often require users to adapt to complex information presentation, as the functions are getting complex and versatile. In addition, current limited modalities of visual and auditory feedback may also overlook the subtle regulation of users’ emotional states.

Haptic feedback, by contrast, offers inherent advantages in addressing these gaps. It typically demands less attentional load and enables more intuitive information perception, allowing users to keep their eyes on the road and thereby reducing potential safety risks. Moreover, haptic feedback does not require public display or presentation—it is quiet and unobtrusive. At the same time, it may tune users’ perception of the environment and their emotional state with subtle adjustments of the physical space, such as dynamic modulation of shape and texture.

Smart Material Technology

Smart material technologies refer to research or inventions of advanced materials or hardware that can programmably change their physical properties as outputs, such as shape, texture, temperature, or color, extending the functionality beyond conventional computing devices. These technologies also provide opportunities for haptic interfaces, enabling material interfaces that are lightweight, flexible, and conformable to versatile environments. Among them, fluid-driven interfaces, such as pneumatic or hydraulic, have been explored as emerging technologies with controllable haptic feedback capabilities.

A Design Case Study: Haptic Steering Wheel

We propose a haptic steering wheel design as a case example to demonstrate an in-vehicle haptic interface enabled by smart materials. The steering wheel incorporates a pneumatically actuated shaoe-changing membrane to provide tactile feedback during driving, supporting functions such as navigation cues, proximity warnings, handover cues from the autonomous driving system, and the regulation of driver drowsiness and stress. We further conduct preliminary user studies to evaluate the performance of the proposed design.

Material Technology

Multiple layers of flexible films are bonded with specific patterns, forming sealed air chambers in the blank areas. When the air chambers are inflated, the internal air pressure increases, raising the films to create tactile feedback. In this work, we consider that haptic feedback is achieved through the deformation-induced stretching of a membrane in a 2.5D configuration. Therefore, the membrane material must possess a certain degree of stretchable elasticity. Accordingly, we select a stretchable silicone membrane for fabrication and use silicone-specific adhesive to bond multiple layers of the membrane together.

Material structure

Material actuation with four different shape-changing patterns

Control hardware using syringe pumps

Steering Wheel Design

We adopt a rectangular steering wheel form and integrate four haptic feedback modules on the inner and outer sides where the hands make contact: INNER-LEFT, INNER-RIGHT, OUTER-LEFT, and OUTER-RIGHT. By coordinating the dynamic inflation and deflation of these four haptic modules, multiple functions can be achieved.

Specifically, the inner haptic feedback regions primarily contact the users’ fingers. As this area of the hand is more sensitive to tactile stimuli, it is well-suited for conveying rapidly changing and semantically complex information, such as navigation instructions. In contrast, the outer haptic feedback regions primarily contact the users’ palms, which are better suited for transmitting slower-changing, simpler, or more ambiguous information, such as the status of the autonomous driving system.

Haptic feedback location design

Interaction function design

Prototype Implementation

To demonstrate and validate the functionality of the proposed haptic steering wheel design, we developed a functional prototype. We first fabricated the haptic modules using pneumatically actuated deformable membrane materials combined with flexible fabric, and then mounted these modules onto a 3D-printed steering wheel frame. The pneumatic tubing was concealed within pre-designed channels inside the steering wheel structure.

Pneumatic happtic button, covered with textiles (3D mesh) for smoothness and protection

Steering wheel design, manufacturing, and assembly

Haptic actuation demonstration

We also considered the engineering feasibility of integrating this design into real-world steering wheel systems. In practical automotive manufacturing, the 3D-printed structure could be replaced by an injection-molded metal frame, while the outer fabric layer could be substituted with leather or other flexible materials. The added haptic layer could replace or partially substitute the conventional padding layer. The pneumatic actuation and control system could be housed within the compartment behind the steering wheel, while the connecting tubes could follow routing strategies similar to those used for wiring of existing steering wheel controls—passing through reserved gaps in the frame, extending to the central hub, and ultimately connecting to the actuation unit located behind the steering column.

Engineering structure and constraints of the steering wheel

User Study

To evaluate the effectiveness of the proposed haptic steering wheel, we used the developed functional prototype and recruited 10 participants for user studies. The experiment consisted of (1) a haptic perception test and (2) a simulated driving test.

User study setup

Study 1: haptic perception test

In the first experiment, we evaluated users’ perception to distinguish haptic feedback across four spatial locations. During the study, each participant received 12 haptic stimuli presented sequentially, including three repetitions for each location. The feedback followed a periodic fluctuation pattern with a 3-second cycle, and the order of the four locations was randomized. Participants were asked to identify the location of each haptic stimulus and report it as quickly as possible. We recorded both the accuracy and response time for each trial.

The results show that the 10 participants completed a total of 120 trials, with 119 correct responses, yielding an accuracy of 99.2%. The average response time for perceiving the haptic location was 2.6 seconds (SD = 1.6 s), with a median of 1.9 seconds. These findings indicate that users can easily perceive and distinguish haptic feedback from the four locations, and can do so quickly and reliably.

Study 2: simulated driving test

In the second experiment, we evaluated the four interaction functions of the proposed design using a self-developed driving simulation environment. In this setup, we created a circular track with designated “key segments.” Visual cues were provided before and after each key segment, allowing participants to experience the four interaction functions as they passed through these sections. Completing one lap of the track took approximately 2 minutes. During the simulation, the vehicle moved at a relatively low and constant speed, and participants were only required to control the steering. To enhance immersion, background music was played throughout the experience. Each participant completed two laps of the track.

After the driving session, participants were asked to complete a questionnaire, rating their experience on a 5-point Likert scale (1 = strongly disagree, 5 = strongly agree) across five dimensions: “clearly perceivable,” “easy to understand,” “low attentional demand,” “helpful for driving,” and “willingness to use.” In addition, we conducted semi-structured interviews to further explore users’ experiences with the haptic steering wheel.

Driving simulation environment that connects to the haptic steering wheel

All participants successfully completed the two laps in the experiment. In the questionnaire results, participants gave the most positive ratings to “clearly perceivable” (M = 4.57, SD = 0.76), “low attentional demand” (M = 4.18, SD = 0.84), and “willingness to use” (M = 4.11, SD = 0.97), indicating that the haptic feedback performed well in terms of perceptibility, minimal cognitive load, and user acceptance.

In the post-study interviews, most participants described the overall experience of the haptic steering wheel as “novel,” “comfortable,” and “engaging.” Some participants also suggested extending the use of haptic feedback to scenarios such as reverse distance indication and lane departure warnings. However, a few participants noted that assigning multiple functions to the same haptic location could sometimes lead to confusion, while others pointed out that overly frequent haptic feedback might become distracting during driving.

Concept Design for In-vehicle Haptic Surface

To extend the scope of haptic surfaces for in-vehicle HMI based on smart materials, we explore other concept designs with a broader application of smart material technologies across the in-vehicle space. We demonstrate the ideas with a series of computer-assisted renderings.