Sitting in a meeting room in a startup office in Lisbon, I silently typed the wordplay to a question only the person opposite would know the wordplay to. What kind of coffee had I asked for when I’d arrived at the office? A short moment later, without plane moving or opening his mouth, the reply came when via a text message: “You had an Americano.”
This wasn’t how I’d expected to spend a Friday afternoon in the city, but here I was, sitting in the offices of enterprise language translation services startup Unbabel, opposite founder and CEO Vasco Pedro, testing what appeared to be a brain-to-computer interface. And it was pretty astounding.
Unbabel’s cadre mission – permitting enterprises to understand and be understood by their customers in dozens of languages long ago led the visitor to think outside the proverbial ‘box’, to develop several projects in-house. It wanted to explore other ways to communicate. Now, as a startup with $90 million in VC funding, yearly revenues of virtually $50 million and having survived the pandemic, Unbabel is doing well unbearable to explore these projects.
Unbabel unveils AI project to give us superhuman capabilities
“We had the idea of looking at brain-to-communication interfaces,” Pedro tells me. “We started doing a tuft of experiments, like a 20% project.” Unbabel’s innovation team is led by Paulo Dimas, VP of Product Innovation which looked into the way our brains evolved.
“You have your limbic system, you have your neocortex. But they’ve unquestionably evolved over millions of years. They’re unquestionably separate systems. And I think what we’re starting to see is scrutinizingly the megacosm of the ‘uber cortex’, which we think is going to be AI-powered, and it’s going to be existing outside of your biological brain,” said Pedro.
Dimas and his team started to squint into Electroencephalogram (EEG) systems, some of which can be invasive to the body. Elon Musk’s Neuralink visitor is famously exploring invasive brain-computer interface devices for humans.
[L to R] Mike Butcher, TechCrunch; Paulo Dimas, Unbabel VP of Product Innovation; Vasco Pedro, Unbabel CEO/co-founder, with the Halo device.
EMG was the gateway
But then Unbabel’s team hit on the idea of using an EMG system. EMG (Electromyography) measures muscle response or electrical worriedness in response to a nerve’s stimulation of the muscle. EMG devices are commonplace and trivial. You can plane buy them on Amazon for a few bucks.
“What we realized was that EEG was still too noisy. We wanted to be non-invasive. But EMG, which measures muscle response, was so less noisy. You can increasingly reliably capture some of the signals,” said Pedro.
The team put sensors in an armband and started to work out what they could measure. “We began to think of EMG as a gateway to smart-ass interaction directly,” Pedro told me.
Then, last year, they decided to vaccinate up an EMG system with Generative AI. Specifically, an LLM which was personalized to the user. But how?
Put simply, the system measured how the wearer of an EMG device would react when thinking of a word. This would help to build up a set of signals which correlated to real words. Feeding those signals into an LLM would midpoint the megacosm of a ‘personalized LLM’.
So when I asked Vasco what kind of coffee I’d asked for via an unseen text message, he was sent those words via an AI voice to his earbuds. He then thought of words like “Black coffee”. The LLM then matched his physical response to the word, checked if he meant “Americano”, then via the audio in an earbud, and then sent the wordplay to me via a text message – in this use case, the Telegram texting app.
The lineage of Halo
Unbabel dubbed its invention ‘Halo’ (after ‘halogram’). An app runs on the wearer’s phone that enables wangle to a inside hub for receiving the liaison and enables liaison with the LLM and responses. The platform is pulling the Open AI ChatGPT 3.5 right now.
Pedro likens Unbabel’s project to driverless car companies hacking together data from normal cameras rather than complicated systems, like Lidar: “We’re gonna get a shit tonne of data, and we can start using it now. We started working four years ago and the tipping point is now in terms of generative AI. This is the moment when this is going to accelerate.”
Admittedly, this isn’t the first time EMG has been used to tenancy a computer and generate responses. For instance, a device made by Facebook-oned CTRL-labs had an EMG wristband in 2019 that picked up on electrical impulses that come from muscle fibers as they move.
Unlocking the locked-in
Unbabel is now working with the Champalimaud Foundation in Lisbon, which works on wide biomedical research and interdisciplinary clinical superintendency in the field of ALS, among many other things. Clearly, though, the system could end up stuff used in other scenarios, such as Cerebral Palsy.
The need for largest interfaces for patients who cannot speak is ongoing. Right now, so-called ‘Alternative and Augmentative Communication’ (AAC) products for ALS sufferers, such as Grid or Tobii, rely on eye-tracking. These systems often require a frustrating scale process for the user, are really only workable indoors, and can be fatiguing to the user. They moreover depend on laboriously slow keyboards.
As Pedros adds: “Our prototype is once stuff endorsed by the major ALS undertone in Portugal. We plan to start deploying this to our first ALS users by Xmas this year. Beyond ALS patients, our current product is moreover relevant for other patients that struggle to type.”
Dimas is moreover now Unbabel’s nominee to Portugal’s newly worked Center for Responsible AI, where he is CEO. This is a partnership with several Portuguese startups and research centers to invest €78 million in AI research, creating 210 jobs under the Portuguese Recovery and Resilience Plan. Partners include Feedzai, Sword Health, Champalimaud Foundation, and others.
Generative AI is coming to wearable devices
Meanwhile, the version of Halo demonstrated to me showed the potential power for Generative AI unromantic to wearable devices. Other teams are exploring this unflinching new world. Just this week neuroscientists were worldly-wise to recreate Pink ‘Floyd’s Another Brick in the Wall, Part 1′ using AI to decipher the brain’s electrical activity.
The concept has been virtually for a long time. In the 1980s, the Firefox movie, directed by and starring Clint Eastwood, posited a world where pilots would tenancy weapons systems via thought-controlled platforms:
