Make codespeed build from fresh git clones every commit

[staticfloat]

To make this feasible I need to:

• Copy out build-intensive deps/ subdirectories such as atlas to save compile time, especially on julia.mit.edu.

This is a note to self.

[pao]

Recloning is pretty inefficient resource-wise. Wouldn't you be better off grabbing the tarball of HEAD of master if you're going to do that? Or directly addressing the git cache corruption issue?

[staticfloat]

Grabbing a tarball could work, except that I want the git information as well, as I use the current git hash, branch, commit date, etc... when recording codespeed metadata. I also am not always grabbing the HEAD tarball; I grab specific git hashes.

I am not at all sure why I'm getting these git errors; in all my interactive usage, I have never run into these errors. My automated git workflow is as follows, for each "flavor" of julia I compile:

pushd $REPOS_DIR/julia-$flavor
git fetch
git reset --hard origin/$BRANCH
git checkout -B $BRANCH $COMMIT
make cleanall
make

If it fails, it most likely means something broke within libuv or something, so I try again:

make distclean
git clean -fdx
rm -rf deps/Rmath
rm -rf deps/libuv
rm -rf deps/openlibm
git submodule update
make

Somehow, this series of commands eventually runs into problems where I get errors such as git unable to read sha1 file of <object hash>.

[pao]

I also am not always grabbing the HEAD tarball; I grab specific git hashes.

Minor quibble; I'm pretty sure you can insert a hash into the Github URL which returns the tarball. Other points are well taken.

I idly wonder if this might be a bug fixed in a newer release of git?

[staticfloat]

Minor quibble; I'm pretty sure you can insert a hash into the Github URL which returns the tarball. Other points are well taken.

You're right; https://github.com/JuliaLang/julia/archive/<git hash>.{tar.gz,zip} works quite well. Neat!

I idly wonder if this might be a bug fixed in a newer release of git?

This would be nice, but the machine this has happened twice on is running 1.8.3.4. I wonder if it isn't a regression on that particular machine though? This may indeed have happened when I upgraded the git version. For now, I'll wait to see if it happens again.

[pao]

Ah yes, the converse "bug introduced in a newer release of git."

[pao]

You might also want to run a git fsck after this happens again, which may help identify exactly what is going wrong.

[staticfloat]

Amazing that in all the searching I did about this problem that command never came up even once. Thanks!

[staticfloat]

While working on an unrelated issue, I figured out why this is happening. OSX automatically deletes files older than 3 days that live in /tmp, which is where I was storing the git clones.

[staticfloat]

While working on an unrelated issue, I think I have figured out the root cause for these errors; OSX automatically deletes files older than 3 days in /tmp. Therefore, the files used to build Julia herself would not get deleted, but old git hash objects would.

Long story short: don't use /tmp for something that should last any reasonable amount of time.