Autonomous toy racing car. CAMaleon team at the Toulouse Robot Race 2017. Humbavision team at IronCar. Medium article: https://medium.com/@araffin/autonomous-racing-robot-with-an-arduino-a-raspberry-pi-and-a-pi-camera-3e72819e1e63
Video of the car: https://www.youtube.com/watch?v=xhI71ZdSh6k
We wrote an article on medium that detailed our approach. You can read it here
En français: http://enstar.ensta-paristech.fr/blog/public/racing_car/
3D models (we used onshape.com):
Note: the Battery Holder was designed for this External Battery
We release the different videos taken with the on-board camera, along we the labeled data (the labels are in a pickle file) for IronCar and Toulouse Robot Race:
For installation, see section Installation.
Compile and upload the code on the Arduino
cd arduino/
make
make upload
Launch the main script on the Raspberry Pi, it will try to follow a line.
All useful constants can be found in constants.py
.
python main.py
python -m teleop.teleop_server
IP
in constants.py
)
python -m teleop.teleop_client
Record a video in the teleoperation mode:
python -m teleop.teleop_server -v my_video
Convert the recorded video from h264 to mp4 using ffmpeg or MP4Box
MP4Box -add video.h264 video.mp4
Split the video into a sequence of images
python -m train.split_video -i video.mp4 -o path/to/dataset/folder
Label the data using the labeling tool: https://github.com/araffin/graph-annotation-tool
Rename the json file that contains the labels to labels.json
and put it in the same folder of the dataset (folder with the images)
Train the neural network (again please change the paths in the script)
python -m train.train -f path/to/dataset/folder
The best model (lowest error on the validation data) will be saved as cnn_model_tmp.pth.
Test the trained neural network (you can use -i
option to test it on a video)
python -m train.test -f path/to/dataset/folder -w cnn_model_tmp.pth
You need a 16GB micro sd card (warning, all data on that card will be overwritten) WARNING: for a smaller sd card, you need to resize the image before writing it (this link and repo may help)
Download the image here
Infos about the linux image: OS: Ubuntu MATE 16.04 for raspberry pi
Username: enstar
Password: enstar
Installed softwares:
Camera and SSH are enabled.
Identify the name of your sd card using:
fdisk -l
For instance, it gives:
/dev/mmcblk0p1 2048 131071 129024 63M c W95 FAT32 (LBA)
/dev/mmcblk0p2 131072 30449663 30318592 14,5G 83 Linux
In that case, your sd card is named /dev/mmcblk0 (p1 and p2 stand for partition).
Write the downloaded image on the sd card.
gunzip --stdout ubuntu_ros_racing_robot.img.gz | sudo dd bs=4M of=/dev/mmcblk0
Enjoy!
The current project is located in RacingRobot/
.
If you want to back up an image of a raspberry pi:
sudo dd bs=4M if=/dev/mmcblk0 | gzip > ubuntu_ros_racing_robot.img.gz
Update your pi
sudo apt-get update
sudo apt-get upgrade
sudo rpi-update
Arduino + Arduino Makefile + rlwrap + screen + MP4Box
sudo apt-get install arduino-core arduino-mk rlwrap screen gpac
OpenCV
All the required packages (except pytorch and torchvision) can be found in requirements.txt
, install them using:
pip install -r requirements.txt
In short:
pip install pyserial tqdm pygame enum34 scikit-learn scipy
ZeroMQ (Message Passing with sockets) for remote control mode
sudo apt-get install libzmq3-dev
pip install pyzmq
Note: for using the serial port, you need to change current user permissions:
# Add user to dialout group to have the right to write on the serial port
sudo usermod -a -G dialout $USER
# You need to logout/login again for that change to be taken into account
Additional python dev-dependencies for training the neural network: On your laptop:
pip install pytorch
pip install torchvision
On the raspberry pi :
And then:
pip install torch-0.4.0a0+b23fa21-cp27-cp27mu-linux_armv7l.whl
Or follow this tutorial: PyTorch on the Raspberry Pi
sudo dd if=/dev/zero of=/swap1 bs=1M count=3072 Status=progress
sudo mkswap /swap1
sudo swapon /swap1
(optional) Install a recent version of cmake + scikit-build + ninja
Install PyTorch
See https://github.com/pytorch/pytorch for dependencies. Additional dependencies:
sudo apt-get install libopenblas-dev libeigen3-dev libffi-dev
# don't forget to set the env variables:
export NO_CUDA=1
export NO_DISTRIBUTED=1
git clone --recursive https://github.com/pytorch/pytorch
python setup.py install --user
# torchvision is not used yet
pip install torchvision --user
OpenCV with Python 2 (RPI), compiling from source (cf Docs and tuto): Additional dependencies:
sudo apt-get install libtbb-dev opencl-headers libomp-dev libopenblas-dev libeigen3-dev
sudo apt-get install libatlas-base-dev gfortran -y
with Gstreamer Then:
cmake -D CMAKE_INSTALL_PREFIX=/usr/local \
-D BUILD_opencv_java=OFF \
-D BUILD_opencv_python2=ON \
-D BUILD_opencv_python3=OFF \
-D PYTHON_DEFAULT_EXECUTABLE=$(which python) \
-D INSTALL_C_EXAMPLES=OFF \
-DINSTALL_PYTHON_EXAMPLES=ON -DBUILD_TIFF=ON -DWITH_CUDA=OFF -DWITH_OPENGL=ON -DWITH_OPENCL=ON -DWITH_IPP=ON -DWITH_TBB=ON -DWITH_EIGEN=ON -DWITH_V4L=ON -DWITH_VTK=OFF -DBUILD_TESTS=OFF -DBUILD_PERF_TESTS=OFF -DCMAKE_BUILD_TYPE=RELEASE ..
OpenCV with Anaconda, compiling from source:
cmake -DPYTHON_EXECUTABLE=/home/ỳour_name/anaconda3/bin/python3 \
-DPYTHON_INCLUDE=/home/ỳour_name/anaconda3/include \
-DPYTHON_LIBRARY=/home/ỳour_name/anaconda3/lib/libpython3.6m.so \
-DPYTHON_PACKAGES_PATH=/home/ỳour_name/anaconda3/lib/python3.6/site-packages \
-DPYTHON_NUMPY_INCLUDE_DIR=/home/ỳour_name/anaconda3/lib/python3.6/site-packages/core/include -DINSTALL_PYTHON_EXAMPLES=ON -DBUILD_TIFF=ON -DBUILD_opencv_java=OFF -DWITH_CUDA=OFF -DWITH_OPENGL=ON -DWITH_OPENCL=ON -DWITH_IPP=ON -DWITH_TBB=ON -DWITH_EIGEN=ON -DWITH_V4L=ON -DWITH_VTK=OFF -DBUILD_TESTS=OFF -DBUILD_PERF_TESTS=OFF -DCMAKE_BUILD_TYPE=RELEASE ..
Info: http://www.hotblackrobotics.com/en/blog/2018/01/22/docker-images-arm/
Build docker image (laptop image):
docker build . -f docker/Dockerfile.cpu -t racing-robot-cpu
Build docker image (raspberry pi image):
docker build . -f docker/DockerfileBase.rpi3 -t racing-robot-rpi3-base
docker build . -f docker/Dockerfile.rpi3 -t racing-robot-rpi3