search

ocaml / opam

opam is a source-based package manager. It supports multiple simultaneous compiler installations, flexible package constraints, and a Git-friendly development workflow.
https://opam.ocaml.org
Other
1.24k stars 353 forks source link

Opam admin cache doesn't create a symlink for sha256 to sha512 #6064

Closed psafont closed 2 months ago

psafont commented 3 months ago

I've run into a couple issues when building xapi's opam repository as an RPM in an environment without internet access:

  1. When running opam admin cache in opam 2.2.0, the sha256 file for crc.2.2.0 is not created, only the sha512 one. The former should a symlink to the latter.
  2. When installing the crc.2.2.0 package, opam 2.1.4 finds that there is no file for its sha256 hash and tries to use internet do fetch it. First from opam.ocaml.org, and after with github.com, after which it fails. I believe it should try to use the sha512 hash to fetch the file from the cache before using other sources.

I'm updating the opam version for issue 2. to see if this behaviour has changed. In any case I've been able to work around issue 1 by manually creating the symlink in the sha256 folder to the sha512 file. Another workaround is removing the sha512 hash from the checksum list: it makes opam create the file created correctly in the sha256 folder.

I don't quite understand why this is the only package in the whole repository with this behaviour, but it looks like a bug.

kit-ty-kate commented 3 months ago

Has issue 2 been fixed in opam 2.1.5 by https://github.com/ocaml/opam/pull/5538 ?

psafont commented 3 months ago

I'll test building an opam 2.1.6 rpm to be used in the build

psafont commented 3 months ago

I've made a github runner to package a repo tarball, it uses ubuntu's opam package: 2.1.2. The resulting tarball contains the file in the sha256 folder, but not a file in the sha512 folder: https://github.com/xapi-project/xs-opam/releases/tag/6.81.0 https://github.com/xapi-project/xs-opam/actions/runs/9763086989/job/26948098641#step:5:1

psafont commented 3 months ago

I've retried building the package with 2.1.6 and issue 2 still happens. Building opam 2.2.0 requires some changes, so it will take some time to test whether it's affected

kit-ty-kate commented 3 months ago

Actually I'm getting the same problem with opam 2.2.0 while using https://github.com/kit-ty-kate/opam-health-check-ng, which means a large number of packages fail and I'm unable to check them.

I'm looking into it.

rjbou commented 3 months ago

Do you have a previously populated cache or it is an new one? Opam creates the symlinks, taking the strongest checksum as real archive and creates links to that one for others checksums. In the case where the cache is already present, especially if the strongest checksum archive file is already present it can alter this mechanism.

psafont commented 3 months ago

Do you have a previously populated cache or it is an new one?

I can reproduce this locally starting from an empty cache. It's strange this only seems to happen for a single package in the repository.

kit-ty-kate commented 3 months ago

we managed to found the issue and fix it in https://github.com/ocaml/opam/pull/6068. We still need to figure out if the fix doesn't introduce any performance issues but at least it's being actively worked on.

psafont commented 1 week ago

I've tested the tarballing with the new 2.3.0 alpha, and can confirm that it builds without issues now, thanks!

rjbou commented 1 week ago

Thanks for the update!