Displays for search and rescue drones


Problem: Drones are used in search and rescue as a fast and cost-effective way to help law enforcement.  There are a few apps available already used to fly the drone and watch video feed, but none are considered to have all features pilots want for this unique task. 

Process: I worked with drone pilots who volunteer their time and resources to help law enforcement with search and rescue missions.  I have conducted interviews and a cognitive walkthrough with one app, Litchi, which most people currently use.  Litchi allows the user to set up a search pattern in a designated area with automated flight that frees up the pilot to watch the video feed for clues about where the lost person may be. 

Interview Results

Based on several interviews with SMEs and pilots, I came up with a set of tasks to include in an app that would allow for a separate screen for a user other than the pilot (a spotter) to watch the video feed for clues of the missing person. Note: The main user is the spotter, but this can be used by the pilot as well.  Tasks begin once the drone is beginning an automated flight pattern - flight set-up has been deemed "good enough" for the purposes of this project based on existing apps. 

1. Flight In-Progress

  • Watch video feed for clues
  • Mark point of interest AND label category if a clue is spotted
    • Tap on screen or voice command
  • Expand map from corner to have a better real time view of where the drone and POIs are

2. End of flight. 

Drone has a small amount of battery left and is hovering at the end point of the pre-determined search area.  The drone will return home automatically once it reaches a certain battery level (set by the pilot in advance).

  • View map with all POIs
  • Review POIs
    • View each POI individually as the screenshot where it was taken
      • Alternatively, view the video clip associated with that screenshot
    • Edit POI
    • Return to POI
  • End flight and return drone home
 

My Design Process

It's easiest for me to get ideas out quickly by drawing rather than on a computer, so I start with white boarding or pencil and paper.  I color code my sketches to keep the different aspects straight and I list out each screen's interactions to make sure I don't miss anything.  In this project, I used several sessions of participatory design to iterate from my first sketches.  Next I took the sketches we came up with together and started prototyping a cleaner version on Balsamiq.  Once I had an interactive prototype, I took it back to the participants and get their feedback before making more changes. 

White boarding session with subject matter expert. 

White boarding session with subject matter expert. 

 

Participatory Design

Since I couldn't set up an in-person meeting with someone in Atlanta, I had to get creative.  I decided to do a participatory design session over Skype, with me sketching on the whiteboard while we talked through requirements and design options.  It turned out to work well, and it was also quite fun!  

I used different colors to code what were my notes, ideas from the participant, questions I needed to ask, interactions that needed to be included, etc.  

Balsamiq was used for wireframing.  For user testing, instead of presenting an interactive prototype, I decided to stay in Balsamiq so I could make immediate changes to the design and confirm with the user. 

Balsamiq was used for wireframing.  For user testing, instead of presenting an interactive prototype, I decided to stay in Balsamiq so I could make immediate changes to the design and confirm with the user. 

 

Prototype #1

User testing and design changes

- Icons - Categories changed to simplify interaction and cover missing areas

- Add declutter button

- Add latitude and longitude

- Add time remaining to battery

 

- Buttons added to view the picture in full screen and view short video clip associated with the image

- Map added showing the bigger picture of which image is being viewed to add context

Final prototype and future work

Next Steps:

  • Develop app to functional version and test in the field
  • More evaluation

Add features:

  • Priority vs category for POI
  • Delete/edit POI
  • Show difference between planned flight and actual flight, if any
  • Choose some or all points to send to pilot to visit on next battery/flight