icecc / icecream

Distributed compiler with a central scheduler to share build load
GNU General Public License v2.0
1.58k stars 250 forks source link

icecc-create-env: add command line --addfile "path target" support #548

Open dmoody256 opened 4 years ago

dmoody256 commented 4 years ago

This allows users to arbitrarily add files that they want to be remapped into some chroot location.

A specific use case is when wanting to remap /proc/cpuinfo from a custom cpuinfo with instantaneous cpu frequency data stripped out for the thinLTO support and the clang bug: https://bugs.llvm.org/show_bug.cgi?id=33008

dmoody256 commented 4 years ago

This feels a bit hackish, both what you need it for and how I assume it needs to be used. How do you actually use this? If you need to explicitly invoke this script this way, for such a workaround, can't you simply unpack the tarball, fix it and repack it?

For example, we want to give the cpuinfo to icecream so we can do ThinLTO incremental linking during the compile remotely, so when it comes back to do the final link all the compiles had the same info as if they were compiled locally. The problem is that we don't want the raw /proc/cpuinfo because it has data in it that is always changing so we end up with a different hash for our compiler package every-time we rebuild. Therefore, we will strip out the data from the cpuinfo with something like cat /proc/cpuinfo | grep -v MHz > ~/mybuild/cpuinfo. However, we want that modified cpuinfo in the correct location, so we want to remap it into the compiler package at /proc/cpuinfo, so then we can call icecc-create-env ... --addfile /proc/cpuinfo=$HOME/mybuild/cpuinfo.

Now of course there are workarounds such as repacking, which is fine, maybe not great. I happened to be looking through the icecc-create-env script and noticed the addfile function did support remapping in some form (although maybe I misunderstood it?), but the command line option --addfile did not, so seemed like a simple fix. I am not an icecream expert, but this worked for me when testing. If you think this approach is hackish, I would defer to your experience. I could write it up as an issue instead.