Fire Fighter

Following Robot

For my senior capstone project, my team worked with the Austin Fire Department to develop a software stack that identifies and follows firefighters around an emergency scene to carry their gear. This was built using ROS2 on a Clearpath Robotics Jackal.

What

  • Used OpenCV and YOLOv11 to identify and track firefighters with an Intel Realsense depth camera.

  • Developed a ROS2 node for identification that would calculate the distance from the robot to the firefighter.

  • That information would be published to the goal_pose topic that gets picked up by our navigation node.

If you’d like to see more about our process and the mechanical design that went with this project, here are the links to our Github and final presentation.

Map of Environment Created by SLAM & Nav2

How

  • Used the depth camera to create a map of the environment as the robot moves using Simultaneous Localization and Mapping (SLAM).

  • Developed a navigation node that would receive the new goal_pose to drive to and send it to Nav2.

  • This node handled the logic of when to accept a new goal pose and how to respond to feedback from Nav2.

  • Nav2 was used to generate the path from the robot’s position to the firefighter, and perform obstacle avoidance based on the data from the map.

  • Created two versions of the following behavior: one that uses Nav2 for path planning and obstacle avoidance, and one that just sends the robot on the straightest path from where it is to the firefighter.

  • The second version performs faster and without hesitation, which could save the firefighters precious time while traversing an emergency site, but will drive directly over objects in its path.

  • Although the second version requires more vigilance during use, it was much more popular with the firefighters.

  • Further research will allow us to refine the Nav2 version and reduce the hesitation time before driving. Some of our plans include adding a behavior tree and the ability to switch between modes for the user.

Github Page
Final Presentation

Video Demo Without Nav2

Video Demo With Nav2

More Information:

Results

Example Image of Tracking Firefighters