Background

Countless homeowners and farmers seek non-lethal and humane autonomous interventions to protect their property from mammalian pests such as squirrels, coyotes, and others. Such a system does not currently exist, and even if it does, it is quite costly for the average homeowners (>$1000). Therefore, our mission boils down to the two following points:

1. Leverage valuable sensors and capabilities of old, unused devices (old smartphones and drones) that homeowners may own
2. Drive out mammalian pests from homeowners’ backyards

This year, HMC INQ developed an autonomous, drone-based system for diverting mammalian pests like squirrels and possums, which can be built with relatively little additional cost and effort. Our simple and elegant system consists of 3 parts:

1. A drone
2. A drone communicator (Cloudifier)
3. A web app to communicate with the user

Additionally, we have explored other potential applications of old devices, such as detecting parking/study space occupancy and remote pet interaction.

Drone / Hardware Devices

Devices

The drone’s flights act as the primary pest deterrent. For this project, our team primarily worked with the DJI Tello drone, which came equipped with a camera, flight capabilities, and an API to control the drone path. 

The goal of the drone is translated to three technical requirements:

1. Controllable, complicated drone path
2. Computer vision
3. Image recognition

To support the complicated drone path, we built on top of the existing DJI Python library to customize different paths.

Then, we developed computer vision capabilities to assist with the drone paths by leveraging the on-board camera, and incorporated it into our flight path so that the paths are autocorrected based on the landmark. 

Lastly, we used color segmentation and image recognition of a landmark (a blue ball) to help the drone center itself over the desired landing spot. The drone recognizes an image with a 3×3 grid, and will adjust its position until the ball is deemed to be in the center of its field of vision (center column or center square). Depending on user settings, the drone will continue tracking the ball, or land at the spot.

In the spring semester, we focused on making our system more robust by making our customized API compatible with other models of drones. Additionally, we added more functionality such as light detection for room occupancy sensors, and motion detection for pet cameras.

Cloudifier

The cloudifier (our own term) is a central device in the full operation of the system that enables a user to remotely control the drone from a web interface. We prototyped a cloudifer with a Raspberry Pi that has the following capabilities:

• Multiple WiFi connection: connecting to both primary WiFi network and drone WiFi network
• Multiple drone connection: support for users with multiple drones
• Control drone’s basic behavior: takeoff, landing, forward, backwards
• Process drone signals: detect light, detect motion, take photos, use computer vision to track objects
• Record sequences of actions and execute

Below is the a picture of our powerful Cloudifier.

Additionally, we built an admin web app was built to communicate directly with the drone, which provided a useful tool for testing and debugging.

Web App

The project’s primary user interface is a web app that serves as the layer of communication between the user and the Cloudifier. The web app is created with Flask, Bootstrap, and Javascript. The goal of the web app is detailed below:

1. Introduce users to the project
2. Collect user settings and preferences
3. Communicate with the Cloudifier
4. Support basic device control
5. Give user real-time feedback (statistics and live view from their devices)

The landing page informs users about the product, while the new device setup page allows users to add their drone’s name, as well as a wifi name and password.

The web app takes on a dashboard interface, with Droneye Live, the main page, allowing users to control the devices, create paths for the device, check battery level and obtain real-time image feedback.

By the end of the year, we were able to successfully demonstrate a low-cost home drone system users can interact with to control and automatically detect specified objects! For more information regarding how we managed the project, please check out the project management page.