GroceryBee

GroceryBee is a robot to assist grocery stores in keeping up with their online orders. GroceryBee will help grocery store employees by collecting the items which are required for an online order from the store shelves, thus speeding up the delivery process. GroceryBee is designed to make decisions in an uncertain environment, so it is able to exploit real-time information from RGB-D cameras to make real-time changes in planning. GroceryBee is able to collect required items in response to the orders while navigating between the supermarket aisles without crashing. It can easily detect requested items on the shelves, picks them up safely and brings them to the delivery point for speedy delivery to customers.

Installation & Running the Project

1) git clone https://github.com/ardaOnal/grocery-bee 2) Install python3, pip, venv.

sudo apt update
sudo apt install python3 python3-pip python3-virtualenv

3) Create a virtual environment in the project's root directory and install the requirements.

python3 -m venv env
source env/bin/activate
pip install -r requirements.txt
pip install torch torchvision
pip install -U git+https://github.com/luca-medeiros/lang-segment-anything.git

pip install python-tsp
sudo apt-get install python3-tk

4) To run the project, type python src/python/main.py
5) To view the output on Meshcat, go to http://localhost:7000 on your browser (the URL may differ depending on which ports are currently in use on your PC so you can also check the console output to see which link to go to, for the Meshcat output). 6) After running the project, you can view the console output for information about various things happening while the project runs (for example the planning of the trajectory, segmentation, perception etc.).

Note: You can switch to the multi-agent branch and run the project to view the simulation where two robots are working simultaneously to fulfil an order.