Could brain waves be the key to advancing physical AI?
Image Credits:Tim Fernholz/TechCrunch / Tim Fernholz/TechCrunch
The Cutting Edge of Physical AI
In a warehouse in San Leandro, California, the world of physical AI is experiencing a unique experimentation: a game of Jenga. The warehouse is home to Encord, a pioneering company that focuses on developing data tools essential for training artificial intelligence models. Andrew Ceja, one of Encord’s robotic trainers, also known as “pilots,” engages in the careful task of removing wooden blocks from a precarious tower. What sets this activity apart is the headset he wears—equipped not just with a camera to track his vision but also with sensors that monitor his brain waves during this task.
Addressing the Data Scarcity Challenge
Encord is part of an emerging group of startups that believes the real challenge in advancing humanoid and warehouse robotics lies not in model architecture, but rather in the significant shortage of real-world physical training data. Unlike companies that merely manage existing data, Encord aims to create and manufacture the data that’s critically needed.
The brain wave headset Ceja is utilizing is developed by Zander Labs, a German firm specializing in neuroscience technology. The idea is that by measuring brain activity, the headset can provide insights into mental states like error recognition, intent, and surprise. Encord’s collaboration with Zander is currently in a trial phase. Their goal is to develop a brain wave-tagged dataset, test it with customer robotics models, and analyze whether it enhances performance before scaling up the project.
Revolutionizing Robot Learning
Lucas Gehrke, a neuroscientist from Zander supervising the project, mentions that tracking brain activity offers valuable clues for model builders, especially when determining the optimal points to employ their most advanced models. This frontier represents a pioneering effort to alleviate the data bottleneck currently faced by robotics.
According to Vineeth Velmurugan, Encord’s head of robot learning and an experienced professional from OpenAI and Berkshire Grey, the company initially aimed to assist in annotating data and evaluating machine vision models. However, as their clients—though undisclosed—ventured into end-to-end robotic manipulation tasks, it became evident that they would need to produce training data in-house. “The data simply does not exist,” Velmurugan explained.
The Inefficiencies of Data Generation
Companies in the robotics sector face challenges as they attempt to leverage generative AI for their models, akin to how it has revolutionized chatbot technology. While self-driving car companies can collect physical-world data, the process is labor-intensive and challenging to scale. Video-based training methods exist but often fall short of delivering the level of detail that real-world data provides. Velmurugan estimates that overcoming these constraints will require data sets roughly five times larger than YouTube’s entire video library—a scale that underscores why generating data has evolved into a lucrative business opportunity rather than merely a research issue.
Harnessing Diverse Data Sources
Organizations focusing on developing robot intelligence are now exploring two primary data sources: “egocentric” video collected by workers with cameras and data obtained from remotely operated robots. Encord is utilizing both methods by extracting egocentric data from various factories worldwide while using its San Leandro facility to experiment with innovative modalities, including brain wave collection.
During a recent TechCrunch visit, pilots employed leader-follower rigs—two paired robotic arms where one replicates the movements of the other—to generate data on tasks such as pouring coffee or stacking poker chips. “Every humanoid company has requested these capabilities,” Velmurugan shared.
The storage areas within the facility were brimming with cartons of simulated training materials, including fake flowers, plastic fruits, and various household items. At one station, pilot Sofia Infante maneuvered robotic arms to plug and unplug Ethernet cables from a server—an essential operation that many data centers hope to automate, albeit with the current limitations of robotic dexterity.
Innovating New Data Modalities
Encord is also exploring a new avenue for data collection by using sensors strapped to the forearm to capture electrical signals from muscles. Traditional video recordings of human hands often miss the complexities of hand movements, but Velmurugan aims to create a 3D model that accurately depicts hand positioning in real-time through arm sensor input, significantly enriching model training.
The datasets generated by Encord are meticulously annotated with specific physical actions, such as “right hand tightens bolt,” to aid large language models in comprehending various tasks. Velmurugan believes that this detailed annotation is worth 100 times more than less specific “egocentric data” for targeted training, even though producing it is approximately 20 times more expensive.
The Economic Shift in Data Production
Despite the apparent value, generating physical training data presents a financial hurdle. Traditional data collection methods—like scraping text from the internet to develop language models—come at minimal costs. In stark contrast, generating physical data is an expensive endeavor, reshaping the economic landscape for developing AI models.
Velmurugan acknowledges important progress in the field, as Encord’s comprehensive perspective allows it to observe trends and techniques that are gaining traction across various robotics companies. This vantage point acts as a significant advantage, providing insights that individual clients may not have.
A Dedicated Workforce In Action
The facility at Encord continues to be a hive of activity as the team develops the foundational elements for neural networks. Both Infante and Ceja are part of a dedicated workforce trained in producing the necessary data for future AI models, coming from previous backgrounds in AI data annotation at companies like Scale. Ceja, who previously worked in waste management assessing robotic performance, finds satisfaction in the diverse challenges encountered in creating robot training tasks. “It’s something new every day!” he states, highlighting the excitement and unpredictability within the field.
Conclusion
The pioneering efforts by Encord represent a crucial step forward in bridging the gap between AI technology and the real world. By focusing on generating the requisite physical training data, the company is uniquely positioned to support robotics firms in tackling the data bottleneck that limits advancements in humanoid and warehouse robotics. As ventures like Encord push the envelope, the potential for enriched, effective training models continues to grow, paving the way for significant innovations in AI and robotics.
Thanks for reading. Please let us know your thoughts and ideas in the comment section down below.
Source link
#brain #waves #unlock #physical
About The Author
