search

alisw / alibuild

A simple build tool for ALICE software
http://alisw.github.io/alibuild/
GNU General Public License v3.0
24 stars 47 forks source link

Protect against reusing partially-uploaded tarballs #794

Closed TimoWilken closed 8 months ago

TimoWilken commented 1 year ago

Previously, if a package's symlinks were uploaded partially (e.g. not all dist symlinks uploaded, but the "main" symlink was present), and that package's hash matched what we need, then its revision number would be reused. This would then fail on upload, as aliBuild would detect a conflict with the partially-uploaded dist symlinks.

Relying on the fact that we upload the tarball last, after any symlinks, we can detect that this has happened by not reusing revision numbers for which we do not already have a tarball available. The syncToLocal function should download any tarball with the right hash, so if one is not available even though a symlink's hash matches, then we know it's probably from a partial upload and we must not reuse its revision number.

Draft since I've not tested this properly so far, I've only run some unittests locally.

@davidrohr this should fix the problem seen in the WeeklyO2 build last night. Though the specific partial upload of QC that was causing problems is now cleaned up -- I must've missed some files last night -- and a rebuild today went fine.

TimoWilken commented 8 months ago

I think this is not needed any more, since aliBuild should detect this another way already.