search

ocaml / opam

opam is a source-based package manager. It supports multiple simultaneous compiler installations, flexible package constraints, and a Git-friendly development workflow.
https://opam.ocaml.org
Other
1.21k stars 347 forks source link

opam does not wait for the Software Heritage fallback archives to be "cooked" #5721

Open kit-ty-kate opened 8 months ago

kit-ty-kate commented 8 months ago

When using the software heritage fallback feature, it is understood that regenerating (as noted in #5720) the archive from a given SWHID (called "cooking" by Software Heritage) might take some time. The code currently denotes this by a 60 seconds timer: https://github.com/ocaml/opam/blob/c4961baf23e681c74b381a5f9c2cfd7362740e68/src/repository/opamDownload.ml#L299

However in practice, trying it locally, leads only a short wait then the following failure:

OpamSolution.Fetch_fail("\"/opt/homebrew/bin/wget -X POST --content-disposition -t 3 -O - -U opam/2.2.0~alpha2 -- https://archive.softwareheritage.org/api/1/vault/directory/6f61d745641e701d5cad73a9e918d5e888d5ab64/\" exited with code 8 \"2023-11-08 16:00:08 ERROR 404: Not Found.\"")

Calling the same command again when the archive has finished cooking, succeeds.

kit-ty-kate commented 3 months ago

The software heritage fallback was disabled by default in https://github.com/ocaml/opam/pull/5899 so moving this issue off the 2.2 milestone

kit-ty-kate commented 1 month ago

After further investigation it looks like wget does not work to fetch the expected data, but curl does:

$ curl https://archive.softwareheritage.org/api/1/vault/directory/6f61d745641e701d5cad73a9e918d5e888d5ab64/
{"fetch_url": ...}
$ wget -O - https://archive.softwareheritage.org/api/1/vault/directory/6f61d745641e701d5cad73a9e918d5e888d5ab64/
<!DOCTYPE html>...
rjbou commented 1 month ago

Some more output


Skipping 121 bytes of body: [{"exception":"NotFoundExc","reason":"Cooking of swh:1:dir:45b87e1ca127914b936884dc73c9f788df5b7abd was never requested."}] done.
2024-06-13 18:04:26 ERROR 404: Not Found.