search

JuliaLang / Pkg.jl

Pkg - Package manager for the Julia programming language
https://pkgdocs.julialang.org
Other
613 stars 251 forks source link

Avoid deleting existing artifacts when ignoring hashes. #3768

Closed nhz2 closed 5 months ago

nhz2 commented 5 months ago

Followup to https://github.com/JuliaLang/Pkg.jl/pull/3764 Fixes bug reported in https://github.com/JuliaLang/Pkg.jl/issues/3643#issuecomment-1905048298

This PR modifies download_artifact to avoid calling mv(src, dst; force=true) where src and dst are both possible valid artifacts.

The main steps of download_artifact are

  1. Return true if the artifact already exists.
  2. Determine the expected artifact path and parent directory.
  3. Make a temporary directory.
  4. download_verify_unpack into the temporary directory.
  5. Calculate the tree hash of the temporary directory.
  6. If the tree hash doesn't match, either throw an error or log an error, depending on the enviroment.
  7. Atomically move the temporary directory to the expected path.
  8. Try to clean up the temporary directory.

If steps 4 to 7 throw an error, that error gets returned, otherwise true is returned.

IanButterworth commented 5 months ago

@staticfloat does this look ok to you?

IanButterworth commented 5 months ago

Given you have a reproducer in https://github.com/JuliaLang/Pkg.jl/issues/3643#issuecomment-1905048298 could that be added as a test?

nhz2 commented 5 months ago

Yes, I'll try and add a test. I should be able to reproduce the issue on Linux using the right environment variables and artifacts.