search

autowarefoundation / autoware

Autoware - the world's leading open-source software project for autonomous driving
https://www.autoware.org/
Apache License 2.0
8.6k stars 2.89k forks source link

feat(docker): re-organize the autoware docker containers #4072

Closed oguzkaganozt closed 4 months ago

oguzkaganozt commented 6 months ago

Description

Re-organize the Autoware Docker Containers for:

Related PRs:

Related links

Tests performed

Notes for reviewers

Some changes might interfere with some Github Actions

Interface changes

No interface changes

Effects on system behavior

No system effect, CI-CD and container runtime improvements made

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

After all checkboxes are checked, anyone who has write access can merge the PR.

mitsudome-r commented 5 months ago

@oguzkaganozt Could you also create PR to autoware-documentation to make sure that install instructions for Docker is valid with this PR?

mitsudome-r commented 4 months ago

The code looks okay to me.

I have noticed that you have tested the action and it seems passing: https://github.com/autowarefoundation/autoware/actions/runs/7905813792

We can test with autoware.universe and other repositories to see if they can use the docker image for the CIs.

mitsudome-r commented 4 months ago

I just noticed that update-docker-manifest is failing. It might be due to docker images that were pushed manually for CES demo... https://github.com/autowarefoundation/autoware/actions/runs/7903383046/job/21571162050

esteve commented 4 months ago

@oguzkaganozt let me know if this is this ready for another review. I saw that there are some conflicts with main, shall I wait until those are resolved to review this PR again? Thanks.

oguzkaganozt commented 4 months ago

@oguzkaganozt let me know if this is this ready for another review. I saw that there are some conflicts with main, shall I wait until those are resolved to review this PR again? Thanks.

@esteve Should be resolved, please review now šŸ‘

xmfcx commented 4 months ago

@oguzkaganozt do you have a PR or a branch on autoware.universe which utilizes the autoware-openadk docker images?

These workflows should be updated to point to the new images:

Once you update them in a branch/PR, and they pass, we can actually modify their origin, which is in:

Those 4 files are synced from autoware_common to autoware.universe with:

From autoware_common, they are also synced to:

:skull: :skull: :skull: I hope I didn't miss any :smiling_face_with_tear:

oguzkaganozt commented 4 months ago

@oguzkaganozt do you have a PR or a branch on autoware.universe which utilizes the autoware-openadk docker images?

These workflows should be updated to point to the new images:

Once you update them in a branch/PR, and they pass, we can actually modify their origin, which is in:

Those 4 files are synced from autoware_common to autoware.universe with:

From autoware_common, they are also synced to:

šŸ’€ šŸ’€ šŸ’€ I hope I didn't miss any šŸ„²

  • @mitsudome-r @oguzkaganozt This is a very very huge scope of work.
  • These images are used by all these repositories.
  • We might want to discuss how to proceed in a meeting.

@xmfcx You're right that the scope of the PR is huge as the container images are widely used on different repos. But I checked autoware_common workflows and they are using ros-humble as a base image(not universe-prebuilt) so they will not be directly affected by this PR. I think we just need to update autoware.universe workflows to make them work with the new container images.

xmfcx commented 4 months ago

@oguzkaganozt Could you update the readme file of each ansible script too?

They are necessary for the manual installation steps.

Install ROS 2 Install ROS 2 Dev Tools Install the RMW Implementation Install pacmod Install Autoware Core dependencies Install Autoware Universe dependencies Install pre-commit dependencies Install Nvidia CUDA Install Nvidia cuDNN and TensorRT Download Artifacts

xmfcx commented 4 months ago

After we resolve all these, we should also test both the .setup-dev-env.sh and the manual installation instructions (with the updated readme and the documentation page PR on separate clean Ubuntu 22.04 installations.

And make sure all the demos work with the updated instructions in both machines.

xmfcx commented 4 months ago

I would like to merge this once this last wave of reviews are resolved.

Thanks @oguzkaganozt @ambroise-arm @esteve @mitsudome-r for your reviews and support for this task!

xmfcx commented 4 months ago

@ambroise-arm could you confirm if you are happy with the changes?

xmfcx commented 4 months ago

I've confirmed that the the images generated by this PR performs well on the Autoware Universe side.

xmfcx commented 4 months ago

Thanks everyone again for your invaluable comments and efforts!

oguzkaganozt commented 4 months ago

I would like to thank everyone again for all of the precious reviews and comments !

HansRobo commented 4 months ago

@oguzkaganozt Thank you for this nice pull-request. build-main and build-main-self-hosted workflows seem to have been left behind at autoware-universe instead of autoware-openadk. https://github.com/autowarefoundation/autoware/actions/runs/8187823403 https://github.com/autowarefoundation/autoware/actions/runs/8187794780

And I tried a quick fix, but the build step in build-main does not work correctly. https://github.com/HansRobo/autoware/actions/runs/8198820592