Unmanned “Unplugged”

Daniel Serrano, ICARUS

Daniel Serrano is a project coordinator at the Integrated Components for Assisted Rescue and Unmanned Search operations (ICARUS) and the head of unmanned systems at ASCAMM in Spain. His team recently worked on a project using UAS for search and rescue and mine detection following major flooding in Bosnia. Increasing Human Potential recently got the chance to catch up with Serrano and learn more about ICARUS and its work in the Balkans.

Can you tell us about the aim of ICARUS and your role in it?

ICARUS is a European research project that aims to develop robotic tools that can assist human crisis intervention teams. The project was awarded funding by the European Commission’s Directorate-General for Enterprise and Industry under the E.U.’s Seventh Framework Program for research and technological development. ICARUS began in February 2012 and is scheduled to end in 2016. The project consortium is composed of 24 partners including universities, research institutes and SMEs [subject matter experts].

In the context of ICARUS, I work as part of the project management team, which supports the Royal Military Academy of Belgium in its role as project coordinator. The various members of the project management team are responsible for managing the project-related tasks that are most closely aligned with their area of expertise.

My primary area of responsibility is leading the work on adaptations to robotic platforms in order to ensure these tools can meet the needs of search-and-rescue teams. Interoperable robotic platforms developed within the context of the ICARUS project can be comprised of up to nine different types of robots tackling different aspects of search-and-rescue operations.

Two main scenarios are targeted by the work of the project: urban search and rescue and maritime search and rescue. Several organizations from different European countries are developing these platforms, which include a solar-powered unmanned fixed-wing airplane, indoor and outdoor rotorcrafts, small and large ground robots, maritime rescue capsules, and surface vehicles.

Credit: Royal Military Academy

I am also responsible for ensuring the interoperability and collaboration among the robotic devices operating as a heterogeneous team, which is a key aspect of the project. The use of assistive unmanned platforms integrated as an interoperable platform will assist crisis management personnel by providing detailed and easy-to-understand information about the situation in question. These systems offer a valuable tool to save human lives, to speed up the search-and-rescue process, and to reduce the risk and impact on citizens during disaster situations.

The ICARUS team was recently in the Balkans during a flood looking for misplaced mines. How were UAS used, and what were they able to accomplish?

The ICARUS UAS mission to Bosnia served the following two main purposes, firstly, assisting the international relief teams by providing an unmanned aircraft system to carry out tasks such as damage assessment, dyke breach detection, mapping and aerial inspection.

The UAS was deployed there to assist the Belgian First Aid and Support Team (B-FAST) in assessing the optimal location to install their high-pressure pumps and in monitoring the water levels after the floods. Despite the installation of the water pumping system, water levels were not decreasing after several days of pumping, due to an undetected dyke breach. The UAS was used to locate this broken dam. Expert analysis indicated that this dam breach could not have been caused by natural means, so the Bosnian Ministry of Justice initiated legal proceedings against the individual(s) who may have caused it and commissioned the UAS images to be used as evidence.

The UAS proved very useful in detecting dyke breaches and in mapping the affected area efficiently. Finding dry land for takeoff and landing was a challenge. As a result, the takeoff and landing was carried out by a trained pilot using a remote control.

After being used to support the work of the B-FAST team, the UAS was also deployed at the request of the German Federal Agency for Technical Relief and Austrian relief workers working in the affected area. These teams requested the assistance of our UAS for aerial inspection, damage analysis, improved situational awareness and for selecting the optimal location for the installation of the high-pressure water pumps.

The second was assisting the Bosnian Mine Action Center in locating explosive remnants of war (ERW) that had been displaced by landslides in the aftermath of the floods. The UAS was deployed to several different regions of the country in order to assist in mapping suspected hazardous areas.

Bosnia and Herzegovina became heavily contaminated by land mines as a result of the war that took place between 1992 and 1995, leaving the country with one of the most serious land mine problems in the world. By the end of the conflict, around two million land mines and unexploded munitions littered the country’s terrain.

The UAS was deployed to conduct inspection flights, especially in areas that were inaccessible to the search-and-rescue teams, due to the high risks. The UAS was used for aerial assessment, to find indicators of mine presence and for mapping of the mine-suspected areas. The UAS helped to generate 3-D maps of the environment that were used to analyze the effects of the landslides on mines and ERWs. Fusing the data acquired by the UAS (3-D digital terrain models) with preexisting data (mine risk maps generated using satellite imagery and data from the mine action centers), it was possible to trace the movement of the land mines and to generate updated mine risk maps as well as maps of mine-affected areas.

To show the impact of the landslides on the mines, it can be reported that some were found up to 23 kilometers away from their original location. This means that the area needing to be searched is huge and that the effectiveness of area reduction techniques such as the use of the UAS combined with 3-D mapping, and tracing the movement of ERWs, thereby limiting the search area, can have a significant impact.

Credit: Royal Military Academy

What benefits did UAS have over other technologies in doing this work?

One of the main advantages of using UAS is that they provide rescue workers with a bird’s-eye view of the area, which enhances their situational awareness dramatically. Nowadays, rescue workers often have to rely on preexisting maps of the disaster-stricken area or satellite data, which is often a few days old. In crisis scenarios like flooding incidents, it is clear that more recent data is required to enable the relief workers to accurately assess the situation (and to predict its evolution). Compared with the exploitation of satellite imagery, using UAS can provide maps in a much timelier manner.

The ICARUS Project is working to use robotics in search and rescue and crisis management. What are the possible applications of UAS in these fields? Why are they suited for this type of work?

UAS can enhance the situational awareness of the operators by providing aerial imagery. In ICARUS, a two-stage approach is followed, experimenting with two types of platforms: fixed-wings and rotorcrafts. The solar-powered unmanned endurance airplane in ICARUS is developed by the Autonomous Systems Lab at the Swiss Federal Institute of Technology in Zurich. Fixed-wing craft are more suitable for undertaking an initial assessment as they cover larger areas in shorter periods of time, due to their ability to fly at high altitudes and at fast speeds. The recorded data may be made available to the ground station in real time or can be post-processed to generate high-fidelity maps of a disaster-stricken area. These models are then used to plan the intervention, select safe locations in which rescue operations can be based, analyze the traversability of roads for the ground teams, divide the area into sectors, (each of which is tackled by an individual SAR team) or other similar applications.

In ICARUS, we also work on two different hovering platforms. An outdoor coaxial quadrotor has been developed by ASCAMM (Spain) and a smaller indoor hexacopter has been designed by Skybotix (Switzerland). Rotorcraft play an important role when the scope is a smaller area, for instance, when search and teams arrive in their designated sector. Flying at a much lower altitude and capable of hovering, these tools provide a much more detailed assessment of a smaller area. For instance, they can be used for area and building inspection as well as the generation of detailed 3-D digital models for the purpose of structural assessment.

All aerial platforms can remain airborne for the duration of the intervention, operating together in synergy. The platforms are fitted with optic and thermal cameras that can be used for locating survivors. The outdoor coaxial quadrotor is able to deploy a survival kit, which is vital when victims are located in remote or inaccessible areas, as rescue personnel will not be able to reach them quickly. The smaller hexacopter is able to navigate indoors and can be used to search for survivors inside buildings.

Can you tell us more about what ASCAMM does and your role in the organization?

ASCAMM Technology Center is a nonprofit private foundation located in Barcelona, Spain. Our mission is to help companies to become more competitive. We fill the gap between basic research in academia and the market. My role in ASCAMM is to lead the unmanned systems group, which forms part of the intelligent systems department. Our activities started back in 2006, under the responsibility of the Aerospace Technology Center, which later merged with ASCAMM in 2012. Our focus is on applied research on autonomous navigation for ground and aerial robots.

As a part of the research and development team for your organization, what are the problems you are currently working to resolve?

Our R&D activities are closely intertwined with aim of the ICARUS project; developing unmanned systems for search-and-rescue missions is an endeavor of the utmost importance, both from a personal and a professional point of view. In relation to the outdoor multirotor, much emphasis is currently placed on the development of the autonomous navigation capability. For instance, developing algorithms for autonomous collision avoidance is very relevant to this scenario. Otherwise, the entire system is dependent on the pilot’s ability to control the device manually. Reliability and safety are also critical, particularly in regard to redundancy as well as wind and water resistance.

In order to streamline operations, we are putting in extra effort in the standardization of protocols and interfaces. Interoperability is a key aspect in the use of robotic platforms and often presents a barrier to the exploitation of unmanned technologies in the field. We are working on the use of standards for the interface between the systems and the centralized command and control station. This should facilitate the integration of different technologies and enhance the reliability of the developments.

From the application point of view, we are researching rapid mapping algorithms. Our end users have indicated that speed is critical in the generation of mapping products that support rescue operations. We are working on different approaches that will enable us to speed up the mapping process, based on multi-sensor, multi-robot data.

Credit: Royal Military Academy

When you approach a humanitarian mission, such as the flood in the Balkans, what are your first steps to be as successful as possible?

Some of the key issues related to the deployment of UAS for humanitarian missions are nontechnical. First, preparedness is key, not just in terms of the UAS itself being ready to go when an emergency call is received, but also that the operations manual, safety manual and pilot certification are in order, as these will be required to obtain flight permits. It is also essential that the required documents for ensuring the UAS can pass through customs have been prepared.

A second key aspect is the tight collaboration with the end users (rescue teams, demining brigades). They will have to voice the request for assistance by the UAS to the authorities, which means that they have to be convinced themselves that the tool is going to help them in their operations.

Another major aspect is the need for collaboration with the authorities. It is imperative to have — prior to any operations — an open discussion with the aviation authorities of the country in which the operations will take place to ensure that both parties understand the rules of engagement, even in a crisis response scenario. The deployment of the UAS in Bosnia-Herzegovina has proven that this is possible. Flight permits that allowed the UAS to fly up to 150 meters across all areas of the territory were granted with the support of the Ministry of Security and the national Directorate for Civil Aviation. Due to the crisis situation, and thanks to the fact that all the required documents for the flight permits were readily available (as they had already been prepared for previous operations), these flight permits were issued within 12 hours following a coordination meeting in the capital, Sarajevo, with the Bosnian Ministry of Security. During this meeting, the UAS mission received the full support of the Federal Civil Protection of Bosnia and Herzegovina, the Ministry of Security, and the Bosnian Mine Action Centre. Such support is vital for the deployment response operations to crisis situations in a foreign country.

Where would you like to see UAS technology applied in the future?

UAS clearly have great potential to be exploited for the benefit of humanitarian operations. Providing support to search-and-rescue, firefighting and other operations deployed to respond to crises is, in my opinion, one of the key ways in which UAS can provide the most added value to society. However, the benefits of these systems are not limited to crises and disaster management. For instance, UAS could also be used for analyzing energy efficiency of a building, inspecting infrastructure and precision agriculture. In the more distant future, one could imagine many different potential applications of UAS; for example, I would hope to see them being used as a means of transport in my lifetime.

Where do you see the future of UAS going, and what is the next advancement that you would like to see?

The UAS sector has experienced notable evolution over the last couple of years as the increased availability of affordable platforms has democratized their use. This is definitely a positive sign for the future of UAS. Hundreds of entrepreneurs could start to use them for commercial benefit, new applications have been developed, and UAS are no longer primarily linked to use for defense purposes.

However, the increase in the widespread use of UAS has caused some issues; people without having an appropriate training or without following standard safety procedures have been able to operate these systems, resulting in the occurrence of some unfortunate incidents. The sustainable development of the sector necessitates the implementation of a formal regulatory framework in countries where UAS are used.

From a technological point of view, I think there will be some hot topics relating to advancement of the current capabilities. Currently a trained pilot is still responsible for operating the flight in most situations. Stress and fatigue experienced by human beings can [have an] impact on the performance of the aircraft. Improved automation will be required to support certain kinds of activities where safety, replication and scalability are required. So far, UAS have been used primarily for providing information that can improve situational awareness. In the coming years, we anticipate more diverse uses of rotorcraft platforms, such as for atmospheric sampling or the delivery of goods.