H2L: Bridging the Gap Between Humans, Machines and Virtual Environments

Imagine being able to control and feel sensations in virtual reality (VR) when playing a video game, for example. For the last ten years, Emi Tamaki has been at the forefront of research on human-computer interaction technologies, exploring how to bridge the gap between the two. Her pioneering AI-based company, H2L (Happy Hacking Life), is developing AI assistive tools to create immersive experiences and realistic sensations to connect physical human feelings with machines and virtual environments, offering users truly unique VR experiences.

Haptics technology creates real-time feedback effects with touch. Rather than relying on the sounds from headphones or images on a screen, haptics technology enables users to experience realistic and immersive sensations when interacting with virtual environments. For example, gaming companies are constantly looking for ways to make their VR games more realistic, developing technologies that can “virtualize” body experiences, allowing players to get a more realistic sense of the position, resistance, and even the weight of objects they interact with virtually.

How it all began

Emi Tamaki, CEO of H2L, has made a significant impact on the field of haptics technology. Her pathway to this industry stemmed from the need to spend several stints in hospital during her teenage years. As a result, she missed out on and became disconnected from everyday life. “I missed out on many life experiences such as participating in barbecues, attending sport games or celebrating family birthdays. The sensation of being absent taught me that life is about embracing experiences and emotions and I wanted to be able to share these experiences. That’s what fueled my drive for innovation,” she says.

Tamaki started researching practical applications of haptics technology at the University of Tokyo where she developed the PossessedHand, a device that controls user's finger motion through electrical stimulation. Using electrical manipulation to stimulate muscles, she demonstrated that PossessedHand could control the movement of 16 finger joints. This was achieved by stimulating the muscles of the user’s forearm with 28 electrodes linked to an electrical pulse generator.

PossessedHand was selected by TIME magazine as one of the 50 Best Inventions in 2011 and featured in the 2022 World Intellectual Property Day Youth Gallery.

Opening up a world of immersive virtual experiences

Since co-founding H2L in 2012, Tamaki has continued to develop haptics technologies for various industries, including the tourism, medical, and sports sectors. H2L created two wearable controllers, UnlimitedHand and FirstVR, which utilize muscle displacement sensors for immersive virtual experiences.

UnlimitedHand is a device designed to transmit human proprioception - the sense that enables us to perceive location, movement and action of different body parts - using Electric Muscle Stimulation. Through electrical stimulation, the motion sensor detects the movements of the arm muscles and transmits them to the computer. Primarily intended for R&D and experimental use, it focuses on the development of BodySharing®, a concept coined by Tamaki that aims to enable humans to share sensations, such as weight and resistance, with other people and with robots through patented muscle displacement sensors.

“After mice, keyboards, speakers, displays, GUIs (Graphical User Interfaces), and HMDs (Head Mounted Displays), touch panels and interfaces have evolved with the times and the amount of information handled, and have greatly changed the industrial structure,” says Tamaki, who believes that UnlimitedHand has the potential to help create the next groundbreaking interface after smartphone touch panels.

FirstVR was created as a specialized controller utilizing sensor technology from UnlimitedHand for research and development and use by the general public. “In order to have such an interface used widely, not only for R&D, but also by the general public, we thought something easier was needed. So we developed FirstVR, a controller that specializes in sensor technology,” Tamaki explains.

FirstVR is considered to be the first controller in the world to use muscle displacement sensors to create a more intuitive and immersive virtual experience. When the controller is wrapped around the arm, the 14-channel optical muscle displacement sensor group detects the movement of the muscles around the forearm, enabling the user to move, manipulate and sense objects with it. FirstVR enables sensory estimation of position, weight, and resistance among the intrinsic senses, as well as force estimation, which allows for many experiences, such as sightseeing and sports experiences, to be uploaded to the computer and analyzed within the metaverse space. FirstVR is a versatile interface that can be applied in multiple industries, just like the display and keyboard of a PC or the touch panel of a smartphone.

H2L plans to industrialize haptics technology and BodySharing, concentrating on developing content applications for experience sharing. “We want experience sharing to permeate our daily lives. For example, we would like to create a society in which ordinary people experience a home run by a big leaguer or a professional musical instrument performance, or surf in Hawaii while in Japan,” says Tamaki.

Development of the haptics market: key challenges

Tamaki is convinced that investment in R&D is central to expanding their market share in the future. “Our investment in R&D enables us to acquire the technical elements necessary to develop our experience-sharing services and remain competitive,” explains Tamaki, adding that the collaboration between H2L's products and the services of various companies has allowed them to develop innovative solutions that users continuously seek.

Innovation in the field of haptics presents its challenges. Critics argue that haptics technology poses risks, for example, if used for criminal activities or warfare. To address this, Tamaki's team has developed products embedded with a system that ceases to function when it senses malicious intent, ensuring these products are not complicit in crime or war. “I believe that one of our current challenges is to make as many people as possible aware of the safety of this technology, because the idea that "haptics technology is dangerous" is a barrier to the spread of the technology,” notes Tamaki.

H2L’s IP approach

H2L employs various intellectual property (IP) rights strategies, in line with its business objectives. In general, however, the company offers preferential access to its IP-backed technology to promote its uptake, foster inter-company collaboration and promote business partnerships. “We view IP as a crucial tool for expanding our vision, not only to ensure the scope of technology enforcement and implementation but also to support open innovation and provide equitable usage among engineers,” Tamaki explains.

Owning IP leads to an improvement in the valuation of the company, and thus plays a role in appealing to investors.

In offering guidance to tech startups, Tamaki stresses the importance of pinpointing the right moment to establish IP in the early stages of research and development. This approach, she notes, not only allows startups to defend their market position, but also paves the way for developing diverse management strategies, such as maximizing IP utilization and enhancing brand recognition and sales efficiency. “We successfully protected our BodySharing interface, which transmits unique senses to humans and computers, with IP rights early on, and have since also acquired IP for analysis and relevant applications,” Tamaki explains.

Tamaki advises that tech startups need to constantly think about IP to ensure their own competitiveness, and to create opportunities for collaboration. “The major strengths of a high-tech startup are its technology and its mobility. On the other hand, sales capacity and stable branding can be weak points. Most startups cannot compete with established companies when it comes to name recognition and sales force. However, as a starting point, we can improve name recognition and sales by using technology. From these points of view, high-tech startups have no choice but to secure intellectual property,” says Tamaki.

High-tech startups have no choice but to secure intellectual property.

Tamaki highlights the importance of building IP awareness within a company at an early startup stage. “Almost all of the main personnel in a high-tech startup company need to have a basic knowledge of intellectual property and an understanding of the company's IP strategy and policy. This includes R&D personnel for IP creation, sales personnel for IP negotiations, general affairs personnel for contract writing and management, and human resources for education and for preventing any IP leaks,” she notes.

She suggests that one way to boost understanding of IP at the company level is to take advantage of free online courses offered by the Japan Patent Office and the WIPO Academy. Additionally, for companies based in Japan collaborating with general support organizations such as the Japan Institute of Invention and Innovation (JIII), which is present in 47 prefectures across Japan, can further enhance their mastery of IP management.