search

randomRexx / ADAS_AI_cart

POZ - Een ADA
1 stars 2 forks source link

readme

ADAS_AI_cart

Made by students of the hogeschool PXL.

The purpose of this project is to develop an ADAS system for an autonomous golfcart.

A Pytorch resnet50-model and openCV library are used to detect objects captured by the Intel T265 Realsense Tracking Camera / Intel D435 Realsense Camera and the TI IWR1642 Radar.

The resulting data is then transmitted to the Jetson-Nano via Json.

The project is programmed on a Ultra96 and/or the zedboard. This is done with ROS2/ROS1 and/or U-boot with C-Kernels to make an application and compare them based on efficiency.

Branches

  Documentation                               general documentation and instructions to install     
  zedboard_ros2_errors_solutions              Errors with installing ROS2 and solutions
  zedboard-C-kernels                          C-test kernels for Ubuntu 18.04 bionical
  zedboard-pytorch-resnet50-obj-detection     Resnet50 python programma with images
  zedboard-Uboot                              U-boot files for zedbord

Flowchart

Project Goals

Zedboard ## Zedboard installation these installation steps include: * 1 [Intel RealSense Camera T265](https://www.intelrealsense.com/tracking-camera-t265/) * 1 [zedboard](http://zedboard.org/product/zedboard) * 1 [TI IWR1642](http://www.ti.com/tool/IWR1642BOOST) **setting up of the VM with ubuntu 18.04** * Follow [the video](https://www.youtube.com/watch?v=QbmRXJJKsvs) to set up [the VM with Oracle's VirtualBox](https://www.virtualbox.org/). * enable USB3.0 in the settings menu by selecting USB and enable the "USB 3.0 (xHCI) Controller"

**setting up of the Realsense T265 Camera** It is recommended to not have anaconda3 installed on your VM since this can create pathing issues and lead to colcon build errors, see the "ros2_errors_and_solutions" to fix these pathing problems without deinstalling anaconda3. * follow [the instructions](https://www.intelrealsense.com/sdk-2/) about the installation of SDK 2.0 in linux Ubuntu 18.04. * Run realsense-viewer to see if the installation was succesfull. * If the Camera is not found between the USB devices, go to the "ros2_errors_and_solutions" branch. this happens because the VM's USB3.0 drivers are having trouble finding the Realsense T265 camera. **setting up ROS2** * follow the instructions on [the Ros website](https://index.ros.org/doc/ros2/Installation/Dashing/Linux-Development-Setup/). it is recommended to install the ros dependencies ("rosdep" command) of both "building ROS 2 linux" and "installation ROS 2 linux". * for colcon building errors go to the "ros2_errors_and_solutions" branch. **Setting up ROS Melodic** * To set up ROS Melodic, follow the instructions on this [link](http://wiki.ros.org/melodic/Installation/Ubuntu). * Type `sudo apt-get install ros-melodic-catkin` * Type `sudo apt-get install cmake python-catkin-pkg python-empy python-nose python-setuptools libgtest-dev build-essential` * Now Catkin is downloaded and the needed dependencies are installed. * To create a Catkin workspace, follow this short [tutorial](http://wiki.ros.org/catkin/Tutorials/create_a_workspace). * After completing the tutorial, the workspace should look like this:

**setting up the TI IWR1642 Radar** * If the Catkin workspace is created it's time to set up the Ros packages. * Go to the home folder. * Type `git clone https://github.com/randomRexx/ADAS_AI_cart.git` * Type `cp -R ~/ADAS_AI_cart/radar/ti_mmwave_rospkg/ ~/[workspace dir]/src/` * Type `cp -R ~/ADAS_AI_cart/radar/serial/ ~/[workspace dir]/src/` * Now the needed Ros packages are in the Catkin workspace. * Go to the workspace. * Type `catkin_make && source devel/setup.bash` * Type `echo "source /devel/setup.bash" >> ~/.bashrc` * Plug in the radar and type in the VM `sudo chmod 666 /dev/ttyACM0` * And `sudo chmod 666 /dev/ttyACM1` * To start the radar, type `roslaunch ti_mmwave_rospkg 1642es2_short_range.launch` * Now the RVIZ is running.

* Open a second terminal. * Type `rosrun ti_mmwave_rospkg Radar_Read.py` * This will print the X, Y, Velocity and Range for every detected object.

**torchVision resnet50 model** * type `sudo apt-get install python3 \ python3-pip \ python3-opencv` * type `pip3 install torchvision` * download the model on the "zedboard-pytorch-resnet50-obj-detection" branch. * type `nano object_detection.py` in de linux terminal to open the python program. * scroll to the bottom where a .png image is requested.

* change the input image to the an .png image of own choice. * save and exit nano and run the "object_detection.py".
Ultra96 ## Ultra96 installation these installation steps include: * 1 [Intel RealSense Camera D435](https://www.intelrealsense.com/depth-camera-d435/) * 1 [Ultra96](http://zedboard.org/product/ultra96-v2-development-board) * 1 [TI IWR1642](http://www.ti.com/tool/IWR1642BOOST) **setting up of the VM with ubuntu 18.04** * Follow [the guide](https://github.com/randomRexx/ADAS_AI_cart/blob/master/InstallatieInstructies%20Petalinux.docx) to set up the Ubuntu for Ultra96. * From here on follow the same steps as discussed in the Zedboard section.

Materials and software

Used Software

Used Hardware

Authors