travis-ci / beta-features

The perfect place to leave feedback and comments on newly released Beta Features.
56 stars 68 forks source link

[Multiple jobs] - Lost apt installed packages despite cache: apt: true #43

Closed wltjr closed 6 years ago

wltjr commented 6 years ago

Please excuse if I am off here or have something misconfigured. I recently switched an overlay to using jobs/stages. All was going well till a new stage, or separate group of stages. The previous apt installed packages were no longer present or the prepared env. Despite having cache: apt: true

You can see the resulting build output here. It failed on the Coverity stage as pip was not installed. I had installed that via the new beta apt integration.

I assumed since I had cache: apt: true, that it would retain anything installed via apt for all jobs/stages. That maybe just keeping apt package cache up to date, vs syncing for each stage not sure. I ended up reverting to old means to install stuff via install.

I noticed another essentially duplicating the packages install section for other job groups. Which I find to be less than ideal. It requires duplication and if you change a package in one and forget or miss the other issues. The legacy install approach uses the same section for all. Though I believe it does install more than once stuff, so is less efficient.

kenman345 commented 6 years ago

Might be off base, but If you look at your builds, you will find that Apt commands run before caching is pulled into the build. So doing it the old way would be the way that it needs to be done if you plan on caching the installed dependencies from apt.

I would like if they were to allow this flexibility with build-stages as it was previously

wltjr commented 6 years ago

No worries, I might have been off base with my understanding of things. I am not familiar with the old ways. I guess what I am after is avoiding some of the env prep. Like installing packages for each of the stages, when all that is the same. Ideally it would just say nuke the travis user home dir and create an empty one. Keeping all installed packages, and any env prep.

For different stages, I understand different env. But for multiple jobs of the same stage. Seems like needless re-duplication of the env. Deploy new container, all that setup, bandwidth wasted, time, etc. Seems like it would be much faster to re-use the container/vm and all installed packages. But I maybe off base. I believe there are ways to achieve such now. Just within the same job, vs multiple.

joshk commented 6 years ago

As Stages is now released and GA, I'm going to close this issue.

If you have any feedback on improvements we should consider, please feel free to add a discussion in our new Community forum: https://travis-ci.community/c/product

Happy building

Josh