Blog

Research Project "DRONES FOR FINDING AVALACHE-BURIED" (D-FAB)

University of Trento Research Units Involved in the Project

 

Department of Industrial Engineering

Department of Information Engineering and Computer Science

Faculty of Law - LawTech Research Group

 

 

Research Team

 

Paolo BOSETTI (Principal Investigator)

Francesco BIRAL 

Daniele FONTANELLI 

Umberto IZZO

David MACII 

Farid MELGANI 

Claudio MIGLIARESI 

Luigi PALOPOLI 

Filippo TRIVELLATO

 


 

A person buried by a snow avalanche can be found by measuring the magnetic field generated by an avalanche beacon or ARVA carried by the victim. However, the signals received are difficult to interpret and require people with good training on the actual searching techniques. Even in this case, though, the find and rescue operation can be significantly slowed down by the typically steep, rough, and uneven ground of an avalanche area. Given the fact that the probability of survival drops from 90% to 30% between 18 and 30 min after the event, any technology that can speed up the spotting of the beacon signal can effectively reduce the death risk for buried people.

We propose to develop a hexacopter drone for helping the rescue team in finding and marking the positions of buried people. Such a drone would carry an ARVA receiver, a collection of systems for mapping, localization, navigation, and attitude/cruise control, and a device for physically marking the position of buried ARVA beacons (e.g., by dropping a flag, by paint-marking, etc.). The rescue team would launch the drone upon arrival on the avalanche scenario and define a geographic boundary for the search area by means of a proper remote interface with the drone controller. As soon as the drone localizes and marks the first beacon, the rescue team can start digging around the first mark, while the drone proceeds in searching for possible other beacon signals. In terms of time efficiency, this scenario has a two advantages: firstly, given that the speed of the drone largely exceeds that of human rescuers—especially considering the uneven and steep ground they are typically moving on—, the beacon localization time can be significantly reduced; secondly, the search for any subsequent buried people can happen in parallel while the rescue team is busy digging, not to mention that a swarm of drones can prove even more effective in compressing the average finding time.

Development of such a system is a highly interdisciplinary task, demanding for competences in mechatronics, automation, control, measurements, signal processing, and tackling of possible legal issues. To date, there are no examples of similar systems on the market, although there are very recent and ongoing research efforts with similar objectives triggered by the recent fast developments in the market of small UAVs. We reckon that this is the right time for starting research activities in this field within our University, for three reasons: i) technology readiness: small robotic drones and UAVs can be realized at reasonable costs; ii) competences: after the recent rearrangement of university Departments, we have now most of the competences of interest in this field in a contiguous space (Povo 2 building, hosting both the Dpt. of Industrial Engineering and the Dpt. of Information Engineering and Computer Science); iii) a large, indoor space suitable to be equipped as a drone testing area has been recently made available to the DII.

 

 


 

D-FAB project aims at developing a vertical takeoff and landing (VTOL) drone for assisting Alpine rescue teams in locating avalanche buried and semi-buried people, with the aim of shortening the localization time thus improving the chances of saving their lives.

The project has to solve a number of problems, relevant to different disciplines:

  1. Drone design. Sturdy, rugged drone mechanical design, suitable for operation in harsh winter alpine conditions. Typical operating conditions—together with the characteristics of the payload needed for victims localization—impact on the system design at different levels, from the power storage, to the resistance against water and humidity, low temperatures, and to the ability to fly in a controllable and stable way even in presence of significant winds as it may be the case on high altitude scenarios. Also thanks to the expertise of DII members in designing and realizing autonomous vehicles and control systems (also considering aerodynamics aspects)[3–6], and in manufacturing structural components in composite materials, it will be realized a prototype of a lightweight, resistant, and foldable VTOL drone structure—which is not yet available as off-the-shelf solution.
  2. Scenario and ego-state reconstruction, by applying sensor fusion techniques to a set of lower level measurements, including GPS location, inertial navigation, height-over-ground detection, apparent wind direction and intensity, ground morphology. More precisely, this will be addressed by developing algorithms that exploit computer vision and image processing techniques to deal with the huge quantity of information that UAV collects (possibly offloaded to a ground station for elaboration) [22,23]. It will consist in the design of a nonlinear filter, which combines image gradient features and Gaussian process (GP) modeling. The image gradient features allow capturing detailed information regarding the structure of the investigated classes of objects (human figures, semi-buried personal items) whereas the GP model fed with image gradient features permits to yield a statistical estimate of the presence of objects of interest for any position within the image. The image/video acquisitions will be performed from predetermined altitude very close to the ground. Accordingly, the GP model will be trained with a library of predefined objects with adequate spatial resolutions. In order to speed up the algorithm, a fast and simple screening technique based on application of adaptive image threshold will be adopted in order to reduce the areas to be analyzed by the detector.
  3. Localization of buried and semi-buried avalanche victims, possibly using standard ARVA beacons together with image analysis and classification, by using the very approach described in B and coherently fused with the other sensed data, for incrementally updating a map of the avalanche area, marking the presence of beacons, items (skis, backpacks), and semi-buried human figures. Regarding beacon signal analysis, there are still few examples of automatic searching. One of the key problems arises from the uncertainty about the transmitter orientation. In fact, the direction of the transmitter antenna changes radically the field shape. From a general point of view, with respect to classical far–field identification problem, in this case we have to identify 6 states for each transmitter, instead of 3, while we can only collect 3 measurements (the H-field vector components). Amongst other solutions based on complex antenna arrangement and circuitry, it is worth recalling the solution proposed in [7, 8]—based upon Bayesian estimation theory and Kalman filters—which is going to be adopted in the project, much likely to prove compatible with the constraints for a drone on-board system.
  4. Navigation. The operating scenario presents harsh conditions and stringent time constraints. Consequently, in our envisioned application, the operators set goals and constraints for the mission and define search areas for the drone. The drone executes the mission autonomously and whenever a meaningful event takes place (e.g., localization of a victim or completion of the scanning mission on the assigned area), it releases markers to pinpoint locations of interest and send notification to the rescue team. During the mission execution operators are relieved of the task of guiding the drone and can shift their attention to other goals (e.g., setting up a protected area for emergency care or dig out a victim that the drone has already located). Mission planning and control in a partially unknown environment with mission critical goals is a difficult task, for which robust and reliable solution are not yet available. The problem is hybrid in nature since the continuous dynamics of the drone is combined to a “logical” formulation of constraints and goals [20] but the hostile environment conditions and the need for flexible response to unanticipated events, requires robust solutions such as can be offered by probabilistic approaches [21] for the specification of the mission and for the search of the solution space. As far as the navigation is concerned, the drone is associated with a set of basic maneuvers (symbols), each one implemented by a suitable control law and associated with a change in the state of the system (e.g., a change in the coordinates). Navigation is implemented by a sequence of symbols decided by the planning algorithms and executed by the control algorithm. The correct sequence is identified by an optimization framework that breaks the high level goals down into a sequence of elementary changes each one implemented by a control symbol. The algorithm will fully exploit the information coming from Task 4 (environmental condition monitoring, surface reconstruction, localization, presence of beacons) and the detected environmental features to plan optimal routes for patrolling the area of interest. According to the stochastic models based on the precious experience of the Alpine Rescue Service as well as on the changing weather conditions (e.g., the presence of sudden wind gusts), the goals and the changes in the state by each symbol have a probabilistic nature. So, the mission planner decides the course of actions that maximizes the probability of fulfilling the goals and meeting the constraints. A mission supervisor monitors the execution of the mission and triggers alternative plans if significant deviations are detected with respect to the planned progress.
  5. Human-Machine Interface. Most of the operational efficacy of our solution lies in the ability for the human operators to easily and effectively and securely define the mission objectives and constraints, and to receive readable and unambiguous notifications of the meaningful events. The harshness of the operational scenario (irregular illumination, presence snow and gales, heavy garments and glass that obstruct the visibility of screen) can put a strain on the most readable interfaces if these factors are not adequately accounted for. What is more, the use of gloves and the low temperature of the fingertips can seriously reduce the precision and the efficacy of using the touch screen. These considerations discourage the straightforward utilization of existing smart phone and tablet apps that have become the natural companion of low cost commercial drones. A close cooperation with a sample of potential users belonging to the rescue teams will allow us to the requirements of the interface, which will be fine-tuned with field tests on potential operational scenarios.
  6. Legal aspects: concerning both legal standards for drones operability and civil and criminal liability rules for controlling and preventing human-induced avalanches. As for the first subtopic, operating a drone poses significant and unprecedented legal issues, considering that the first attempt in regulating drone flight is extremely recent (the first Italian regulation of this new aerial technology was issued by ENAC in 2013 and the UE is currently preparing a uniform discipline across member states -http://europa.eu/rapid/press-release_IP-14-384_en.htm). More specifically, the issues to be considered are: role of the civil aviation authority over drone operations; safety and manufacturing standards as well as product liability rules; rules of civil liability applicable to drone-induced damages in the event of mid-air collisions or surface crushes; privacy issues, since data collected by UAVs must comply with the applicable data protection rules, and data protection authorities must monitor the subsequent collection and processing of personal data. As regards the second subtopic, a vast debate flourishes in legal literature today on the civil, criminal and administrative rules aimed at controlling the risk of avalanches, since the diffusion of the practice of ski ride and extra slopes skiing (also encouraged by winter tourism marketing strategies) increases the statistical occurrence of avalanches and calls for the application of more severe rules to skiers engaged in such practice, also inducing a serious concern for the social allocation of the costs associated to the rescuing of avalanche-buried skiers. This debate needs to be analyzed and reconstructed in order to provide the research project with an in-depth analysis of the law and economics of avalanche prevention, which is the field targeted by the innovative product at the core of the research project.

Clearly the operational efficiency of the proposed system strongly depends on the integration of these interdisciplinary aspects, as it is typical for mechatronics devices. For these reasons, the team proposing this research has been selected in order to provide a set of competences encompassing all the problems A.–F.

 

 


 

 

Paolo BOSETTI (Principal Investigator)

Francesco BIRAL 

Daniele FONTANELLI 

Umberto IZZO

David MACII 

Farid MELGANI 

Claudio MIGLIARESI 

Luigi PALOPOLI 

Filippo TRIVELLATO

T1 REQUIREMENTS AND SPECIFICATIONS

Requirement and specifications for both the drone and the testing facility are defined and collected, also according to the Quality Function Deployment approach. During this phase, the Advisory Board and the Alpine Rescue Service of the Autonomous Province of Trento are involved in defining requirements and use-case scenarios. Particular care is paid in defining the power and payload weight budget for the drone, taking into consideration statistics on survival time expectancy for buried people and number of buried people per avalanche. Also, a set of reference tests to be performed when assessing the prototype effectiveness is here defined.

Results: A Req&Spec manual is released after month 2, and later on it is updated when needed.

Members of the research team in this task: ALL.

  1. Drone Architecture

According to Task 1 outcome, this task carries out the design and development of the VTOL drone. Whenever possible and meaningful, off-the-shelf subsystems and components will be adopted, in order to minimize the development time. Prototyping facilities of the Mechatronics Group (Bosetti and Biral) as well as expertise in developing composite structures (Migliaresi) are exploited for realizing custom components and structures. Also, aerodynamics modeling and simulation expertise is available within the DII (Trivellato)

It is also investigated—and possibly implemented—the separation of some computational demanding, (but non time-critical) operations, which can be offloaded to a ground station (as the laptop/tablet controller) in order to limit the weight and power consumption of the drone.

A critical point is the definition of a hardware software architecture that permits an easy integration of hardware and software components. We will adopt a layered approach, with a bottom layer composed of low-power hardware (e.g., PIC or Arduino microcontrollers) used to control sensors and actuators and interconnected by a reliable bus infrastructure (e.g., CAN), and a top layer embedding all the computing hardware required for the processing of visual data, and for planning and control functions (which could be based on a quad-core ARM board). The software layer will build on top of an infrastructure consisting of a real-time operating system (most likely a real-time variant of the Linux Kernel) and of a lightweight middleware to enable an easy integration of software components.

Results: a working prototype drone, refined enough to allow the evaluation of its performance: maneuverability, stability, speed, energy efficiency, compatibility with mountain harsh environment, transportability.

Members of the research team in this task: Biral, Bosetti, Fontanelli, Migliaresi, Trivellato

  1. Mapping and Navigation

This Task is in charge of designing two main functions for fulfilling the desired goal: a) understanding the surrounding environment, b) locating possible victims, and c) autonomous navigation. 

According to the approaches discussed in the Project description, this central task hosts the development and preliminary testing of subsystems dedicated to ARVA beacon localization, self-localization and mapping of the surrounding environment, and algorithms for navigation and control of the drone. These activities are carried out on a functional subsystem approach: after designing a high-level architecture that defines the communication standards and interfaces, the three above named functions are individually developed and tested before integration.

Results: ARVA beacon signal processing algorithms for buried localization; image processing algorithms to define the context; a navigation algorithm for drones acting in harsh environments.

Members of the research team in this task: Melgani, Biral, Fontanelli, Palopoli

T 4 Smart Sensing

This task focuses on designing suitable sensing techniques that are specifically tailored to support drone navigation in avalanche areas. The presence of severe slopes or large obstacles (e.g., trees, rocks) and, more in general, the roughness of the surface due to the rubble transported by the avalanche itself, can indeed make drone navigation much more difficult than in usual scenarios. These problems are further emphasized by the strong requirement of flying steadily at a limited height (e.g., a few meters) above the avalanche to maximize the probability to detect the signal broadcasted by the ARVA device. 

The task on smart sensing will consists of the following subtasks:

  • Analysis and selection of different types of sensors to select the most suitable devices. Besides a GPS receiver coupled with an Inertial Measurement Unit (IMU) with a small form factor, an ultrasonic or a laser-based system are envisioned to scan the surface and to measure in real-time and with high accuracy (a few centimeters) both height-over-ground and distance from possible obstacles. A camera will be also included for context refinement and enhanced diagnostic capabilities in synergy with Task 3.
  • Development of a data fusion algorithm (e.g., an extended Kalman filter or an unscented Kalman filter) to estimate the 3-D position of the drone in the avalanche area. The area of search will be defined through the HMI. The algorithm should be robust even when the GPS data are not available.

Resultslist of sensors to be integrated on the drone; localization and position-tracking algorithm tested and characterized through simulations and preliminary experiments.

Members of the research team in this task: Macii, Fontanelli

  1. Human-Machine Interface

A main, tablet-based HMI is developed, allowing the drone operator to give a set of high-level directives and to access drone status information. In particular, the HMI must provide a way for quickly defining the boundary of the research area, the initial search direction, the patrolling height-over-ground, and other navigation parameters, and possibly to see on a local GPS-based map the location of beacons, updated as the drone localizes them. Effectiveness and viability of human spoken language interface (via portable radios that are common equipment for rescue teams) is also investigated and possibly implemented.

Results: prototype software to be deployed on tablet devices for controlling the drone.

Members of the research team in this task: Palopoli, Bosetti

  1. Test Facility

The hangar located in Pergine Valsugana, which is in the availabilities of DII, is equipped with devices and structures in order to allow indoor testing of drones in perfect safety for researchers and in conformity with the recent national regulations (which make relatively difficult outdoor testing). The hangar is a 25m x 40m space with a free height exceeding 20m. A large, yarn mesh surrounded volume is delimited within it, leaving small corridors around it for access, protecting the drone(s) from hitting people, equipment, or walls/ceilings. An inclined yarn mesh on the testing volume floor holding polyurethane foam blocks of various shapes can be used for simulating the uneven surface of an avalanche area. A suitable indoor localization system is selected and installed, in order to allow the simulation of a GPS-system and also providing a reliable ground-truth, essential for the testing phases. Other systems and infrastructures, as arrays of RGBD cameras also used for image recognition, WiFi networks, RFID beacons, are possibly also implemented, according to results of Task 1.

Results: a testing facility represented by a yarn-mesh surrounded isolated volume of about 20x30x20m3, equipped with 3-D localization system and other ground-based communication, computation, and measurement systems, according to results of Task 1.

Members of the research team in this task: Fontanelli, Biral, Bosetti, Macii, Melgani

  1. Integration

Subsystems of Tasks 3, 4, and 5 are integrated on the drone prototype resulting from Task 2. The resulting prototype is also integrated with subsystems and instrumentations of the testing facilities (with particular regards for the indoor localization system).

Results: A complete drone prototype, able to autonomously navigate within the testing facility, to localize an ARVA beacon, and to mark its position both physically (e.g., by a flag) and as its 3-D coordinates via HMI.

Members of the research team in this task: Bosetti, Fontanelli, Macii, Melgani, Migliaresi

  1. Testing

The prototype drone is run against the set of tests defined in Task 1, where it has to localize an ARVA beacon in the testing facility. The figure of quality is an index encompassing the time for localization, the accuracy and the repeatability of each localization, and the failure rate. The same tests are also performed by human ARVA operators to provide a reference benchmark.

Results: a set of statistics, showing how the prototype drone can improve the effectiveness of buried localization w.r.t. human ARVA operators.

Members of the research team in this task: Biral, All

  1. Legal analysis, Legal compliance, Interaction with Stakeholders, Dissemination

This task is collateral to all the other tasks of the project, and will cover the entire 18 months duration of the project, being divided in 2 phases: 12 months dedicated to legal analysis of issues in the Italian and European context; the following 6 months focusing on the legal compliance of the project, a task that will ease the dissemination of the project results with the external institutions involved and the stakeholders interested to the industrial exploitation of the project (CAI, Alpine Rescue Service, Police operators, ski area operators industry). The first phase will entail comparative research and critical evaluation of the legal sources applicable to drone operation and avalanche safety; and will culminate with the preparation of a study report. Building on the results of the first phase, the second phase will check the legal compliance of the project, envisaging internal seminars with the investigators responsible for the main task of the project, as well as workshop with stakeholders and end-users representatives, like the Italian Alpine Club (CAI), its local section SAT, and the Alpine Rescue Service, and ski area operator industry representatives, who will improve the solidity of the project assumptions with their knowledge about the real experience in carrying out a rescue operation for avalanche buried people.

Finally, An Advisory Board is appointed within this task, and involved in project by meeting with the project steering committee every six months. Also, fundraising and sponsorship opportunities are evaluated with the aim of providing continuity and successful exploitation during and after the project end.

Results: contributions to definition of requirements in Task 1, and continuous improvement throughout the project. Dissemination activities (papers, conferences, final event). Guidelines for legal aspects.

Members of the research team in this task: Izzo, Bosetti, Macii, Migliaresi