feat(ci,docker): save `RUN --mount=type=cache` ccache on GitHub Actions cache

youtalk commented 2 weeks ago

Description

[ ] https://github.com/autowarefoundation/autoware/pull/4842

I've finally understood the BuildKit doesn't preserve cache mounts in the GitHub Actions cache by default. If we wish to put our cache mounts into GitHub Actions cache and reuse it between builds, we can use a workaround provided by reproducible-containers/buildkit-cache-dance

https://docs.docker.com/build/ci/github-actions/cache/#cache-mounts

I found that the buildkit-cache-dance action does not work effectively unless the setup-buildx-action action is executed beforehand, so it was temporarily moved from docker-build-and-push/action.yaml to the beginning of each workflow file. This will be improved in the next PR.

Tests performed

~Currently, as part of autoware.universe is broken,~ I show the build results of only the core module. https://github.com/youtalk/autoware/actions/runs/9462979885?pr=44

1st try (no cache):
- docker-build-and-push-main (no-cuda) 10m 7s
- docker-build-and-push-main (cuda) 16m 9s
2nd try (with cache):
- docker-build-and-push-main (no-cuda) 4m 17s
- docker-build-and-push-main (cuda) 5m 27s

Effects on system behavior

Not applicable.

Interface changes

Pre-review checklist for the PR author

The PR author must check the checkboxes below when creating the PR.

[x] I've confirmed the contribution guidelines.
[x] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers

The PR reviewers must check the checkboxes below before approval.

[x] The PR follows the pull request guidelines.

Post-review checklist for the PR author

The PR author must check the checkboxes below before merging.

[ ] There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

xmfcx commented 2 weeks ago

@youtalk I think after this PR was merged, docker-build-and-push-main-self-hosted.yaml started failing.

https://github.com/autowarefoundation/autoware/actions/runs/9477822140/job/26113063357#step:9:859

  /usr/bin/docker buildx bake --file docker/docker-bake.hcl --file /home/ubuntu/actions-runner/_work/_temp/docker-actions-toolkit-MB8IMw/docker-metadata-action-bake.json --file /home/ubuntu/actions-runner/_work/_temp/docker-actions-toolkit-Jw8xni/docker-metadata-action-bake.json --file /home/ubuntu/actions-runner/_work/_temp/docker-actions-toolkit-zQNIl8/docker-metadata-action-bake.json --set *.platform=linux/arm64 --set *.args.ROS_DISTRO=humble --set *.args.BASE_IMAGE=ros:humble-ros-base-jammy --set *.args.SETUP_ARGS= --set *.args.LIB_DIR=aarch64 --set *.cache-from=type=registry,ref=ghcr.io/autowarefoundation/autoware-buildcache:cuda-arm64-main --set *.cache-from=type=registry,ref=ghcr.io/autowarefoundation/autoware-buildcache:cuda-arm64-main --set *.cache-to=type=registry,ref=ghcr.io/autowarefoundation/autoware-buildcache:cuda-arm64-main,mode=max --provenance false --push --print
  unknown flag: --provenance
  See 'docker buildx bake --help'.

Do I need to install something to the arm64 machine to make this work? It is a self hosted machine and I have access to it.

xmfcx commented 2 weeks ago

I've updated the machine and maybe it will start working again :man_shrugging:

https://github.com/autowarefoundation/autoware/actions/runs/9516463540

youtalk commented 2 weeks ago

@xmfcx Thank you! I think we need to update docker and you did it.

xmfcx commented 2 weeks ago

Yes, it seems the system upgrade solved it. I've also restarted and reinstalled the actions runner to have it clean.

autowarefoundation / autoware