The horrific aftermath of the recent earthquakes that struck Syria and Turkey makes it clear why a range of technologies are needed to search for survivors in the rubble of destroyed buildings. Now, researchers at the Polytechnic University of Madrid have developed an autonomous robotic system that combines artificial intelligence with multispectral imaging to search for people in dangerous and difficult to access post-disaster environments.
The system uses an Altum multispectral camera that operates at five wavelengths – red, green and blue visible light; near infrared; and red edge light. The camera is attached to a remote-controlled quadruped robot that can be deployed in dangerous environments such as a collapsed building.
Christyan Cruz Ulloa and colleagues developed multispectral indexes that can be used to identify people within rubble. An index is a way of combining images taken at different wavelengths to enhance the ability of the multispectral imaging system to identify specific objects. Such indexes are often used to identify vegetation in satellite images, but no index had existed for identifying people in a rubble environment.
Convolutional neural networks
Once the team had identified an index that was suitable for both indoor and outdoor identification of people within a background of rubble, the next step was to train several convolutional neural networks (CNNs) to identify human forms – such as a torso, head and hand – within rubble. The researchers then compared efficacy of the different CNNs. This allowed them to select the best CNN for this application – a CNN called YOLOv5m.
The researchers then compared their prototype technique with previous systems, which tend to use either red-green-blue or infrared imaging to locate victims. They conclude that their multispectral system has greater versatility when it comes to locating people in both indoor and outdoor environments.
The team describes the system in a paper that has been accepted for publication in Machine Learning: Science and Technology.
