Engineers at RMIT University have invented a small ‘neuromorphic’ device that detects hand movement, stores memories and processes information like a human brain, without the need for an external computer.
Team leader Professor Sumeet Walia said the innovation marked a step towards enabling instant visual processing in autonomous vehicles, advanced robotics and other next-generation applications for improved human interaction.
“Neuromorphic vision systems are designed to use similar analogue processing to our brains, which can greatly reduce the amount of energy needed to perform complex visual tasks compared with digital technologies used today,” said Walia, Director of the RMIT Centre for Opto-electronic Materials and Sensors (COMAS).
The work brings together neuromorphic materials and advanced signal processing led by Professor Akram Al-Hourani, who is Deputy Director of COMAS.
The device contains a metal compound known as molybdenum disulfide, or MoS2.
In their latest study, the team showed how atomic-scale defects in this compound can be harnessed to capture light and process it as electrical signals, like how neurons work in our brain.
“This proof-of-concept device mimics the human eye’s ability to capture light and the brain’s ability to process that visual information, enabling it to sense a change in the environment instantly and make memories without the need for using huge amounts of data and energy,” Walia said.
“Current digital systems, by contrast, are very power hungry and unable to keep up as data volume and complexity increases, which limits their ability to make ‘true’ real-time decisions.”
The research is published in Advanced Materials Technologies. Walia and Al-Hourani are corresponding authors and Mr Thiha Aung, a PhD scholar at RMIT, is first author.
RMIT has filed a provisional patent for the work.
Seeing the future in the wave of a hand
During experiments, the device detected changes in a waving hand’s movement, without the need to capture the events frame by frame – this is known as edge detection, which requires significantly less data processing and power.
Once the changes were detected, the device stored these events as memories like a brain.
The researchers conducted experiments in the spectrum visible to the human eye, which built upon the team’s previous neuromorphic research in the ultraviolet domain.
“We demonstrated that atomically thin molybdenum disulfide can accurately replicate the leaky integrate-and-fire (LIF) neuron behaviour, a fundamental building block of spiking neural networks,” Thiha said.
The past UV work only involved the detection, memory making and processing of still images. In both the visible-spectrum and UV devices, memories could be reset so that devices were ready to perform the next task.
Potential applications
The team’s innovation could one day improve response times of automated vehicles and advanced robotic systems to visual information, which could be crucial particularly in dangerous and unpredictable environments.
“Neuromorphic vision in these applications, which is still many years away, could detect changes in a scene almost instantly, without the need to process lots of data, enabling a much faster response that could save lives,” Walia said.
“For robots working closely with humans in manufacturing or as a personal assistant, neuromorphic technology could enable more natural interactions by recognising and reacting to human behaviour with minimal delay,” Al-Hourani said.
Next steps
The team is now scaling up the proof-of-concept single-pixel device to a larger pixel array of MoS2-based devices.
The Australian Research Council has recently funded the team with a Linkage Infrastructure, Equipment and Facilities (LIEF) grant to enable this scaling up of their neuromorphic devices.
“While our system mimics certain aspects of the brain’s neural processing, particularly in vision, it’s still a simplified model,” Walia said.
“We will optimise the devices to perform specific real-world applications with more complex vision tasks, and further reduce power consumption.”
The team plans to develop hybrid systems that integrate their analogue technology with conventional digital electronics.
“We see our work as complementary to traditional computing, rather than a replacement,” Walia said.
“Conventional systems excel at many tasks, while our neuromorphic technology offers advantages for visual processing where energy efficiency and real-time operation are critical.”
The team is also investigating materials other than MoS2 that might extend capabilities into infrared, which could enable real-time tracking of global emissions and intelligent sensing of contaminants such as toxic gases, pathogens and chemicals.
‘Photoactive monolayer MoS2 for spiking neural networks enabled machine vision applications’ is published in Advanced Materials Technologies (DOI: 10.1002/admt.202401677).