Facebook has taken a look at its plans for a new interface for augmented reality, based on technology from CTRL-Labs, the startup it acquired in 2019. In a video, it shows wristbands that use electromyography (EMG) to translate subtle neural signals into actions – such as typing, fighting or playing games like an archery simulator. The tires also provide haptic feedback, creating a system that responds to basic hand tracking options.
Facebook Reality Labs published a blog post which sets out his work on a prototype of the wrist straps. In the simplest way, the groups can follow basic gestures that Facebook calls ‘clicks’, which are supposed to perform reliably and easily. They’re a bit like the everyday Microsoft HoloLens “Air crane” gesture but detected with the nerve signals running along your arms, rather than visual sensors mounted on a headset.
However, the bands can theoretically do much more. For example, they can track the nerve signals your brain sends to your fingers as you type, allowing you to type on a virtual keyboard without physical buttons. And unlike a regular keyboard, the straps can slowly adjust to the way you type – so they can learn how to move your fingers when you make regular typing mistakes, and then automatically correct them and instead determine what you probably meant. . .
That would be a big change in the way most people deal with computers, but conceptually this is not a major update on how CTRL-Labs described its work years ago. In fact, the ultimate possibilities for EMG wristbands are much more thought-provoking: finally, you can perform the same typing gesture types by think to move your hands instead of really moving it. Facebook wants to further streamline user interactions by relying on artificial intelligence and augmented reality glasses, with which they announced last year that they were working.
Even in their simpler versions, these controllers offer an interface that you can carry all the time instead of picking up and holding, like the current Oculus Touch VR controllers. The effect may be similar to smaller boot offers like the Mudra Band, which senses gestures via an Apple Watch band.
One important new addition is haptics. Facebook says it is implementing different prototypes that can give you subtle feedback using different methods. One, the ‘Bellowband’, has eight pneumatic bellows around each wrist. It can be inflated or deflated in patterns that produce clear sensations. Another is ‘Tasbi’, which uses vibrating drives and a ‘new wrist knife mechanism’. When combined with visual feedback from an AR headset, they can provide a wealth of information through a simple and intuitive interface.
Facebook insists that although the group reads neural signals, “it is not similar to mind reading.” This is how it explains the concept:
You have many thoughts and you choose to act on only some of them. When this happens, your brain sends signals to your hands and fingers to tell them to move in specific ways to perform actions such as typing and swiping. It’s about decoding the signals on the pulse – the actions you’ve already decided to perform – and translating them into digital commands for your device.
CTRL-Labs still characterized this technology as a brain-computer interface, but it is in stark contrast to technologies such as Elon Musk’s Neuralink – which reads neural activity directly from the brain through an implant. Implants have unique uses, especially for people with paralysis or amputated limbs, whose bodies simply cannot send nervous signals to a wristband. But Facebook CEO Mark Zuckerberg recently criticized the implant as a short-term consumer technology, saying that “we do not think people want to have their heads drilled to use virtual or augmented reality.” Wristbands also do not have quite the same scare factor for privacy as something that reads your mind at the source.
That said, the bands will almost certainly collect a lot of data. This can include incredibly fine variations in typing patterns; overall levels of physical stress; and any biometric information captured by fitness sensors, augmented reality glasses and other technologies that can be integrated with the tires. (Facebook Reality Labs notes that it has a “neuroethical program” that explores the implications of privacy, security and safety of AR and neural interface technology.)
Like most wearable technology, EMG tires offer an intimate look at how our bodies move – and while it may not sound quite as creepy as a tire that reads your mind, it still requires a lot of confidence.