hercules-ci / support

User feedback, questions and our public roadmap. help@hercules-ci.com
5 stars 1 forks source link

Take care with the naming of source checkouts #40

Closed michaelpj closed 3 years ago

michaelpj commented 4 years ago

I noticed that I was getting far fewer cache hits than I'd expect on Hercules.

It turns out that you really want to set name if you're using one of e.g. the nixpkgs cleanSource functions, otherwise you can end up with the name of the current folder in the parent directory: https://github.com/NixOS/nixpkgs/blob/master/lib/sources.nix#L65

Now, this is bad enough as it is, and probably enough to mess up sharing your cache between Hercules and your local machine. But it gets worse: the sources I see from hercules have names like this: /nix/store/nlhnmg58c8q569ndh0bmdqn5667i1kbg-input-output-hk-plutus-e12feab--language-plutus-core--language-plutus-core

I think what this means is that Hercules is checking out my source into a folder that includes the name of the current revision in it... which means that we are guaranteed a cache miss for everything that depends on the source on every commit :scream:

(I'm slightly guessing here, but I can't see where else that bit of sha would come from.)

Now, users should just Not Do That, but it's very easy to do, and I think Hercules could mitigate this significantly.

michaelpj commented 4 years ago

To be clear, what I'm suggesting is just not including the revision in the folder name (if that's what's happening), or making it part of the parent folder or something.

michaelpj commented 4 years ago

For bonus points, people are most likely to have checked out the repo foo/bar as bar on their local machines, so having it in a folder called bar is most likely to end up getting cache hits if they've made this mistake.

roberth commented 3 years ago

I will be retrieving the bonus points when version 0.8 is released ;)

roberth commented 3 years ago

Has been released for a while but is now also in the stable branch.