save the planet 2/2: multi-stages prod build: lighter and faster

[rvalyi]

This PR makes prod builds an order of magnitude faster and lighter

Currently, in the prod mode, every time the Acsone Odoo bedrock base image gets a security update (once a week), all the Docker lines caches are swept and we rebuild the world, downloading 15 years of Odoo git history that weights 5.7 Gb. It can easily take 20 min in the cloud to 1h30 with a bad onsite connection...

With this change, we pack the bare git-autoshare object references minimum to make the build always fast and light: git history objects are never copied in the production image, only the <250 Mb worktree is. The auxiliary base image weights also no more than 280 Mb because we use fetch --shallow-since the release date: so we can get the Odoo PR merges without carrying the whole "fatty odoo" git repo.

Cached builds still take less than 3 seconds while builds after an Acsone Odoo bedrock update will not download anything from odoo/odoo, only copy the 250 Mb local worktree. And odoo-spec,yaml updates will not trigger large downloads either, only the git delta will be fetched. Most of these builds don't exceed 2 minutes.

Only a standard Dockerfile, no host hack. The auxiliary images are built here https://github.com/akretion/odoo-docker-builder

see https://docs.docker.com/build/building/multi-stage/ full explanation and real life example https://gitlab.akretion.com/akretion/dualsun/-/merge_requests/179

cc @hparfr @PierrickBrun @sebastienbeau @bealdav @renatonlima @mbcosta

[bealdav]

I'm very interested to have build faster because currently it be can fast or really slow depending on cache availability I suppose At the current time I have an emergency fix and CI is blocked to

#8 0.401 (INFO) [09:32:17] git_aggregator.repo  src   Start aggregation of /odoo/src
#8 0.401 (INFO) [09:32:17] git_aggregator.repo  src   Cloning git repository https://github.com/odoo/odoo in /odoo/src
#8 1.103 ============ USE GIT AUTOSHARE ==========
#8 1.141 Initialized empty Git repository in /root/.cache/git-autoshare/github.com/odoo/

If your patch can reduce this time I'm really OK, but I'm expert to guess if it's OK or not.

[rvalyi]

I'm very interested to have build faster because currently it be can fast or really slow depending on cache availability I suppose At the current time I have an emergency fix and CI is blocked to

#8 0.401 (INFO) [09:32:17] git_aggregator.repo  src   Start aggregation of /odoo/src
#8 0.401 (INFO) [09:32:17] git_aggregator.repo  src   Cloning git repository https://github.com/odoo/odoo in /odoo/src
#8 1.103 ============ USE GIT AUTOSHARE ==========
#8 1.141 Initialized empty Git repository in /root/.cache/git-autoshare/github.com/odoo/

If your patch can reduce this time I'm really OK, but I'm expert to guess if it's OK or not.

It absolutely fixes it! In the very worst case with no cache at all and after security updates in the base image it will take less than 4 minutes. When the host has the auxiliary builder image, it takes never more than 2 ou 3 minutes and will only download small deltas from odoo/odoo even if you change your odoo-spec.yaml or after security updates in the base image. If you only touch local-src, of course it takes less than 3 seconds.

I changed something however compared to my initial PR: the auxiliary builder images now use a single branch fetch of the Odoo serie along with the OCA/OCB delta fixes directly in /odoo/src ( see https://github.com/akretion/odoo-docker-builder/blob/main/16.0/Dockerfile ) and I don't use git-autoshare anymore at all for the odoo/odoo build. I explain here why: https://github.com/akretion/odoo-docker-builder/tree/main#odoo-builder-images

I think we could even further optimize the build of /odoo/external-src too (but not using git-autoshare here either along other things). But it would make less difference. What really sucks is odoo/odoo and this is what I fixed here.

Also this new line in your odoo-spec.yaml is extremely important https://github.com/akretion/docky-odoo-template/pull/92/files#diff-f0f5e9142bbdd2f98b2b238237c9ee728147f89595a69bf60e9f6573ff7e6de8R2 to avoid git-aggregate to fetch all Odoo branches since 15 years ago again.

You can see a real life example here with the fast builds, even when I swept the cache on purpose: https://gitlab.akretion.com/akretion/dualsun/-/merge_requests/179

cc @hparfr @PierrickBrun @sebastienbeau @bealdav @renatonlima @mbcosta

[sebastienbeau]

I am going to review this in november