robodrome commented 1 year ago

Checklist

[X] I've read the contribution guidelines.
[X] I've searched other issues and discussions Q&A category. I found no duplicate issues.
[X] I'm convinced that this is not my fault but a bug.

Description

After starting autoware stack in docker and AWSIM on same host, everything seems to work, but AUTO button in RVIZ remains grayed out.

Expected behavior

After setting 2D Goal Pose in RVIZ, the AUTO button should be ready to be enabled. Normally the vehicle would start moving after pressing it.

Actual behavior

After setting 2D Goal Pose in RVIZ, the AUTO button flashes and then remains grayed out. The route is planned and RVIZ shows everything is ok. These ROS messages keep repeating. I have attached a log file. logfile.txt

Steps to reproduce

Follow QuickStartDemo instructions here.

Use autoware docker container instructions here.


export ROS_LOCALHOST_ONLY=1
export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp

if [ ! -e /tmp/cycloneDDS_configured ]; then sudo sysctl -w net.core.rmem_max=2147483647 sudo ip link set lo multicast on touch /tmp/cycloneDDS_configured fi

rocker --nvidia \ --x11 \ --user \ --privileged \ --network=host \ --volume $HOME/projects/autoware/autoware \ --volume $HOME/projects/autoware/nishishinjuku_autoware_map \ --volume /tmp \ -- ghcr.io/autowarefoundation/autoware-universe:humble-latest-cuda


```bash
export ROS_LOCALHOST_ONLY=1
export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp
export RCUTILS_COLORIZED_OUTPUT=1

cd ~/projects/autoware/autoware
source install/setup.bash
ros2 launch autoware_launch e2e_simulator.launch.xml vehicle_model:=sample_vehicle sensor_model:=awsim_sensor_kit map_path:=$HOME/projects/autoware/nishishinjuku_autoware_map

Start AWSIM.

NOTE: autoware was built inside docker container without errors. CUDA support enabled.

Versions

Ubuntu 22.04
ROS2 Humble
Autoware: main (humble)
AWSIM v1.1.0 (linux)
NVIDIA-SMI 515.86.01 (host)
CUDA: 11.7 (host)

Possible causes

I am not sure. Everything seems to be working. Before I enabled tensorrt modules, the car would move and follow the route. It would not detect traffic lights and predict movement of other actors, but it moved and I could press the AUTO button. After compiling autoware with tensorrt it fails. Same result on other machine and same result when not using docker but native install.

Additional context

By looking at the error messages (see log in Actual Behavior section), I can't find out what exactly is the issue here. I do not have reference log to compare it against. Perhaps someone could have a look at the log and understands what could be the problem here?

shmpwk commented 1 year ago

@robodrome

Thank you for your concrete description. Could you try this docker https://github.com/tier4/AWSIM/issues/81 ?

autoware docker container instructions here

The docker you mentioned is for pure Autoware. Unfortunately it does not officially support AWSIM simulation.

Looking at your log, there is not a critical error, including tensorrt related error.

robodrome commented 1 year ago

@shmpwk

Thank you for your response.

Could you try this docker https://github.com/tier4/AWSIM/issues/81 ?

This is a docker container for AWSIM. I am running AWSIM natively on the same host as autoware container. Having said that, I want to create a docker-compose to launch the whole stack to make things easier. I will share it when I get it to work.

The docker you mentioned is for pure Autoware. Unfortunately it does not officially support AWSIM simulation.

Could you tell me what autoware docker image works with AWSIM? I used the same docker source as mentioned in issue 81. When I look at the instructions here, I notice that I need to use branch awsim-stable. I chose main which is humble. Could this be the problem? In other words, is this documentation up-to-date? UPDATE: I tried this version awsim-stable which results in same issue.

Looking at your log, there is not a critical error, including tensorrt related error.

Good to hear. Thanks for checking. This means that these errors should probably be warnings. Or that certain configurable thresholds should be altered. I will investigate further and contact Autoware developers about this.

Thanks again for your help!

shmpwk commented 1 year ago

@robodrome

When I look at the instructions here, I notice that I need to use branch awsim-stable. I chose main which is humble. Could this be the problem?

main branch is up-to-date but not thoroughly tested yet. awsim-stable branch has been tested many times and expected to work reliably. We strongly recommend using awsim-stable branch. Note that there are huge gap between them so that I recommend that you clean build awsim-stable branch from the start to eliminate the cache.

robodrome commented 1 year ago

@shmpwk

Thank you. I did remove the whole autoware directory and cloned into awsim-stable. After manual building I saw the same issues. Only thing left I can think of is that I use a docker image to build and run. This one. I then use rocker to start the container as recommend by the autoware docker instructions. Is this docker container (from awsim-stable branch) method supported/tested?

I also used the docker build script. After executing this script, there is no install directory to source install/setup.bash. This method seems broken.

I will list all steps:

Steps to reproduce

Prepare

install ubuntu 22.04

install nvidia GPU drivers (515)

sudo apt remove nvidia-*
sudo apt-get remove --purge '^nvidia-.*'.
sudo apt autoremove
sudo apt install nvidia-driver-515 nvidia-dkms-515
sudo reboot
nvidia-xconfig --query-gpu-info

install docker-ce instructions
install nvidia container toolkit instructions

install extra packages

sudo apt update \
&& sudo apt-get install --yes \
git \
gpg \
vim \
wget \
tmux \
screen \
libarchive-tools # bsdtar for piped unarchiving (no tmp file, faster)

export AUTOWARE_ROOTDIR=$HOME/projects/autoware

Build autoware container (awsim-stable branch)

cd ${AUTOWARE_ROOTDIR}
git clone --depth=1 https://github.com/autowarefoundation/autoware.git --branch=awsim-stable
cd ${AUTOWARE_ROOTDIR}/autoware
./docker/build.sh

NOTE: This script doesn't seem to build some segmentation packages because DOWNLOAD_ARTIFACTS is disabled by default. Does this means that e.g. traffic light detection is disabled by default using this method? Affected packages: tvm_utility, lidar_centerpoint_tvm, lidar_apollo_segmentation_tvm

Get map data

cd ${AUTOWARE_ROOTDIR}
wget -qO- https://github.com/tier4/AWSIM/releases/download/v1.1.0/nishishinjuku_autoware_map.zip | bsdtar -xvf-

Run autoware stack inside docker container

if [ ! -e /tmp/cycloneDDS_configured ]; then
    sudo sysctl -w net.core.rmem_max=2147483647
    sudo ip link set lo multicast on
    touch /tmp/cycloneDDS_configured
fi

rocker --nvidia \
--x11 \
--user \
--network=host \
--volume ${AUTOWARE_ROOTDIR}/nishishinjuku_autoware_map:/autoware_map \
--volume /tmp \
-- ghcr.io/autowarefoundation/autoware-universe:humble-latest-cuda

Inside container:

export ROS_LOCALHOST_ONLY=1
export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp
export RCUTILS_COLORIZED_OUTPUT=1

source install/setup.bash

ros2 launch autoware_launch e2e_simulator.launch.xml \
vehicle_model:=sample_vehicle \
sensor_model:=awsim_sensor_kit \
map_path:=/autoware_map

There is no install directory to source install/setup.bash. This method seems broken.

robodrome commented 1 year ago

I have used the docker container I built before to build the autoware source awsim-stable.

cd /autoware
rm -rf src/ build/ install/
mkdir src
vcs import src < autoware.repos

source /opt/ros/humble/setup.bash
rosdep update
rosdep install -y --from-paths src --ignore-src --rosdistro $ROS_DISTRO

MAKEFLAGS="-j`nproc`"
colcon build \
--symlink-install \
--cmake-args \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_CXX_FLAGS="-w" \
-DDOWNLOAD_ARTIFACTS=ON \
--no-warn-unused-cli

Note the DOWNLOAD_ARTIFACTS flag which enables downloading the image segmentation models. And the --no-warn-unused-cli flag to suppress cmake warnings for unused variables.

It works like before, but it still will not move. It seems to me like image segmentation detects an occlusion_spot between the car and the crossing/trafficlight. I added two images: one with green traffic light (single occulusion spot detected), the second with two occlusion spots (unknown spot and red/yellow traffic light.) See screenshots below.

NOTE! When I do a clean colcon build without the -DDOWNLOAD_ARTIFACTS=ON flag, everything works fine. Without image segmentation that is, which makes it run through red lights and making things hazardous for other traffic.

Any ideas on this? Could this be an issue with autoware perception model(s)?

rviz_screenshot_2023_02_15-10_37_51 Single occulsion spot (traffic light)

rviz_screenshot_2023_02_15-10_36_59 Two occulsion spots (traffic light + unkown in front of vehicle)

robodrome commented 1 year ago

For some reason it seems to work now. I will look into this and try to find out what went wrong.

robodrome commented 1 year ago

It works now. I do not exactly know what went wrong. These are the steps that worked for me. I tested this on two different Ubuntu 22.04 machines. I had to enable the DOWNLOAD_ARTIFACTS flag to make use of image segmentation features. Hope it helps someone:

Build autoware docker image

Prepare

sudo apt update \
&& sudo apt-get install --yes \
libarchive-tools # for bsdtar

export AUTOWARE_ROOTDIR=$HOME

Build docker image

NOTE: This container will be used as a build container for autoware and to run the demo.

cd ${AUTOWARE_ROOTDIR}
git clone --depth=1 https://github.com/autowarefoundation/autoware.git --branch=awsim-stable

cd ${AUTOWARE_ROOTDIR}/autoware
./docker/build.sh

Build autoware (inside container)

Start container:

rocker --nvidia \
--x11 \
--user \
--privileged \
--network=host \
--volume ${AUTOWARE_ROOTDIR}/autoware:/autoware \
--volume /tmp \
-- ghcr.io/autowarefoundation/autoware-universe:humble-latest-cuda

Inside container:

# this is an external - mounted - directory, you only need to build once
cd /autoware
mkdir src
vcs import src < autoware.repos

MAKEFLAGS="-j`nproc`"

colcon build \
--symlink-install \
--cmake-args \
-DCMAKE_BUILD_TYPE=Release \
-DCMAKE_CXX_FLAGS="-w" \
-DDOWNLOAD_ARTIFACTS=ON \
--no-warn-unused-cli 

# close container
exit

Get simulator (Tier4 AWSIM)

Download and extract

cd ${AUTOWARE_ROOTDIR}
wget -qO- https://github.com/tier4/AWSIM/releases/download/v1.0.1/AWSIM_v1.0.1.zip | bsdtar -xvf-
chmod +x ./AWSIM_v1.1.0/AWSIM.x86_64

Get demo map data (Nishi-Shinjuku, Japan)

cd ${AUTOWARE_ROOTDIR}
wget -qO- https://github.com/tier4/AWSIM/releases/download/v1.1.0/nishishinjuku_autoware_map.zip | bsdtar -xvf-

Run demo

Run autoware (shell 1)

if [ ! -e /tmp/cycloneDDS_configured ]; then
    sudo sysctl -w net.core.rmem_max=2147483647
    sudo ip link set lo multicast on
    touch /tmp/cycloneDDS_configured
fi

rocker --nvidia \
--x11 \
--user \
--privileged \
--network=host \
--volume ${AUTOWARE_ROOTDIR}/autoware:/autoware \
--volume ${AUTOWARE_ROOTDIR}/nishishinjuku_autoware_map:/autoware_map \
--volume /tmp \
-- ghcr.io/autowarefoundation/autoware-universe:humble-latest-cuda

Inside container:

export ROS_LOCALHOST_ONLY=1
export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp
export RCUTILS_COLORIZED_OUTPUT=1

source /autoware/install/setup.bash

ros2 launch autoware_launch e2e_simulator.launch.xml \
vehicle_model:=sample_vehicle \
sensor_model:=awsim_sensor_kit \
map_path:=/autoware_map

Wait a minute for autoware to start before launching the next step.

Run AWSIM (shell 2)

NOTE: This must be done on the same host machine.

export ROS_LOCALHOST_ONLY=1
export RMW_IMPLEMENTATION=rmw_cyclonedds_cpp
export RCUTILS_COLORIZED_OUTPUT=1

# redundant, this has already been done on this host
if [ ! -e /tmp/cycloneDDS_configured ]; then
    sudo sysctl -w net.core.rmem_max=2147483647
    sudo ip link set lo multicast on
    touch /tmp/cycloneDDS_configured
fi

# make sure AUTOWARE_ROOTDIR is set
${AUTOWARE_ROOTDIR}/AWSIM_v1.1.0/AWSIM_demo.x86_64

tier4 / AWSIM

Quick Start Demo - AUTO button can not be enabled in RVIZ #91

Checklist

Description

Expected behavior

Actual behavior

Steps to reproduce

Versions

Possible causes

Additional context

Steps to reproduce

Prepare

Build autoware container (awsim-stable branch)

Get map data

Run autoware stack inside docker container

Build autoware docker image

Prepare

Build docker image

Build autoware (inside container)

Get simulator (Tier4 AWSIM)

Download and extract

Get demo map data (Nishi-Shinjuku, Japan)

Run demo

Run autoware (shell 1)

Run AWSIM (shell 2)