mestrinio/PSR_Trabalho3

[![Contributors][contributors-shield]][contributors-url]

PSR - Trabalho Prático 3

THOR - Trully Hardworking Outstanding Robot

Repository Owner: Pedro Martins 103800
Collaborators: Luís Fernandes 103085 & Afonso Pereira 89142

This repository was created for evaluation @ Robotic Systems Programming "PSR 23-24 Trabalho prático 3".

Make Suggestion

Table of Contents

About the assignment
Objectives
Getting Started
- Setup
Usage

Object Spawn
Detection
Specific Missions/Features

Contact
Acknowledgments

About the Assignment

This assignment was developed for Robotic Systems Programming. It uses a simulated environment to program a simulated robot, model TurtleBot3. The bot is meant to be accomplish a certain number of missions, including detection of objects or people, using image similarity or database comparison (with YOLOv8), and other navigation and localization missions. The model has been modified to have similarities with the "Thor" god character. The program uses ROS and its programmed with Python language and OpenCV library. You can see some videos of THOR in the link bellow.

This repository was created for evaluation @ Robotic Systems Programming "PSR 23-24 Trabalho prático 3".

Thor missions

(back to top)

Objectives/Missions

Robot configuration (Objective)

The robot model is based on Turtlebot3 Waffle Pi but has been customized to fit the Thor theme. Colors and 3D features have been altered and included to match Thor's outfit style, and his hammer. An extra camera has been added to the top of the model.

Scenery Mapping (Objective)

A previous mapping of the simulated house has been done using the procedure taught in class.................

Robot navigation (Mission)

This robot features 4 different navigation modules:

Teleop driving (incrementation of velocity values, linear and angular)
Teleop videogame driving (WASD controls the instantaneous velocities)
Autonomous point-to-point at specific location selected in RViz
Autonomous point-to-point at specific room selected in mission manager.

Object spawning in scenery (Objective)

It is possible to spawn different objects in different positions using a specific script.

Object Detection (Missions)

THOR has it's own object detection features, which include usage of color and shape to detect objects using it's cameras.

Specific Missions

A more detailed view of the missions and features is explained later on this guide.

Getting Started

To use THOR, you need to have ROS properly installed on your system ( ROS Installation ). Furthermore, you must have turtlebot3 original model associated ( Turtlebot3 Models ). Lastly, the house needs to be downloaded to the previously made catkin_ws/src directory (git clone https://github.com/aws-robotics/aws-robomaker-small-house-world) and some other packages must be installed on python for everything to work. IMPORTANT NOTE: YOLOv8 from ultralytics occupies a lot of storage so be sure to have at least 20 GB free to use.

Setup

Installs

sudo apt-get install ros-noetic-map-server
sudo apt-get install ros-noetic-amcl
sudo apt-get install ros-noetic-navigation
sudo apt-get install ros-<your-ros-distro>-cv-bridge
sudo apt-get install ros-<your-ros-distro>-image-transport
sudo apt install python3 python3-tk
pip install numpy
pip install ultralytics
pip install gTTS
pip install playsound
pip install opencv-python

(back to top)

Usage

Launch simulation (Follow order)

roslaunch robutler_bringup_tp3 gazebo.launch
roslaunch robutler_bringup_tp3 bringup.launch
rosrun robutler_bringup_tp3 mission_manager_menu

Navigation (choose one):

rosrun robutler_bringup_tp3 teleop_key
rosrun robutler_bringup_tp3 teleop_keypressteste

Keybindings:

'W' to move forwards
'S' to move backwards
'D' to turn right
'A' to turn left

Other navigations:

Using RViz select option to move to 2d point.
Right press the balloon over THOR (as seen in the picture bellow) to open missions menu, and select option to move to desired room.

***

Object Spawn

The user is able to spawn various predefined objects in various spots in the house by using the following structure rosrun robutler_bringup_tp3 spawn_object.py [-l LOCATION] [-o OBJECT]. The command would look something like:

rosrun robutler_bringup_tp3 spawn_object.py -l on_bed -o sphere_v

Bellow you can find two lists, containing both the code and respective name of all the usable locations and objects.

List of spawnable locations

- on_bed (on the bed); - on_bed_side_table (on the bed side table); - on_bedroom_table (on the bedroom table); - on_kitchen (on the kitchen floor); - on_living_room (on the living room floor); - on_playground (on the playground floor); - on_living_room_table (on the living room table); - on_living_room_table2 (another spot on the living room table);

List of spawnable objects

- sphere_v (Violet Sphere); - sphere_r (Red Sphere); - sphere_g (Green Sphere); - sphere_org (Orange Sphere); - sphere_Y (Yellow Sphere); - sphere_b (Blue Sphere); - cube_r (Red Cube); - cube_b (Blue Cube); - person_standing (Person Standing up); - laptop (Open Laptop); - can_coke (Can of Coke);

Detection

By analising the images captured by it's cameras, THOR is able to detect a plethora of things. By selecting "Search for object in division" in the mission menu, the user will be able to choose an object for THOR to find in a given division. The object can be balls and cubes of various colors, a coke can and even a laptop. It will also count how many objects of the specified type were found in it's search!

THOR's detection module is also equiped to handle more advanced missions, such as checking if the table is clean, which will check if there are any objects on the table, or if someone is home, which will take him around the house searching for a Human being.

Alternatively, if you want to test your colorblindness, THOR will help you with that! The following code will allow you to drive around the house, seeing the house through his lenses as he points out where certain colors are present in the picture at any time!

rosrun robutler_bringup_tp3 color_detection.py

### Specific Misions/Features *** Have you ever wanted to see how your roomba would act if you left it charging for a whole month? Well, THOR will allow you to satisfy this desire, while also providing a first view! By selecting the mission "F1 Mode?", THOR will start racing around the room as if it's life depended on it! Not a fan of high speed races where you narrowly avoid all obstacles? Well then, why not...going against them, then? If you select the mission "Touch object", THOR will touch a green ball, trying compensate for the user's lack of touching grass. Are you alone? THOR will speak to you!! Everytime you select a mission or that it's state changes, he will keep you informed. If you're really bored, he will even tell you a story! Just ask him to in by using the mission menu.

(back to top)

## Contact Luís Fernandes - luis.c.fernandes8@ua.pt Nuno Afonso Pereira - nafonsofp@ua.pt Pedro Martins - pedro.mestre@ua.pt Project Link: [Trabalho Prático 3](https://github.com/mestrinio/PSR_Trabalho3)

(back to top)

## Acknowledgments * Professor Miguel Oliveira - mriem@ua.pt

(back to top)

[contributors-shield]: https://img.shields.io/github/contributors/RobutlerAlberto/RobutlerAlberto.svg?style=for-the-badge [contributors-url]: https://github.com/mestrinio/PSR_Trabalho3/graphs/contributors [product-screenshot]: docs/logo.png