search

vericast / conda-mirror

Mirror upstream conda channels
BSD 3-Clause "New" or "Revised" License
72 stars 60 forks source link

[BUG] Problematic implementation of the mirror #47

Open ericdill opened 7 years ago

ericdill commented 7 years ago

Through reviewing #45, I've discovered a bug in the current implementation of the mirror. The problematic aspect is as follows:

  1. _validate_packages can remove packages on disk that exist in the local repodata.json file
  2. The local repodata.json file is not updated before that problematic file is removed
  3. This means that a user of the condaserver can ask for a package that conda thinks it has (since it is still in repodata.json) but that it can't find (because it's been removed from disk)

As such, this code needs to be changed so that

  1. _validate_packages returns a list of packages to remove
  2. a new dict of package metadata is created that does not contain the packages that we are going to remove
  3. That new package metadata dict is written to disk as an atomic operation
  4. The problematic files are removed

With the above changes we will reduce the chance that the user will encounter an error from conda saying that the file cannot be found on the conda server. This is not a blocking issue on getting this PR merged. I'll fix this problem in a follow-on PR