search

ocaml / opam

opam is a source-based package manager. It supports multiple simultaneous compiler installations, flexible package constraints, and a Git-friendly development workflow.
https://opam.ocaml.org
Other
1.23k stars 351 forks source link

Use of rsync for local repository #1961

Closed silene closed 9 years ago

silene commented 9 years ago

I have a local repository (a local checkout of the coq/repo-stable repository):

$ opam repo
  10 [local]   coq-repo     /home/guillaume/src/opam-coq-repo
   0 [http]     default     https://opam.ocaml.org

I already have used this setup successfully some times ago (but perhaps not with opam 1.2), but now when I try to install a package described in this repository, I get the following behavior:

$ opam install coq:interval
The following actions will be performed:
 - install   coq:flocq.2.4.0                       [required by coq:interval]
 - install   coq:ssreflect.1.5.0                   [required by coq:interval]
 - install   coq:interval.2.0.0
=== 3 to install ===
Do you want to continue ? [Y/n] 

=-=- Synchronizing package archives -=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
[WARNING] Rsync partially failed:
  rsync: link_stat "/home/guillaume/src/opam-coq-repo/archives/coq:interval.2.0.0+opam.tar.gz" failed: No such file or directory (2)
  rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1183) [sender=3.1.1]
[WARNING] Rsync partially failed:
  rsync: link_stat "/home/guillaume/src/opam-coq-repo/archives/coq:ssreflect.1.5.0+opam.tar.gz" failed: No such file or directory (2)
  rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1183) [sender=3.1.1]
[WARNING] Rsync partially failed:
  rsync: link_stat "/home/guillaume/src/opam-coq-repo/archives/coq:flocq.2.4.0+opam.tar.gz" failed: No such file or directory (2)
  rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1183) [sender=3.1.1]
[ERROR] Could not download archives of coq:flocq.2.4.0, coq:interval.2.0.0 and coq:ssreflect.1.5.0

When using the --debug flag, I see that opam tried to use rsync:

rsync_file src=/home/guillaume/src/opam-coq-repo/archives/coq:flocq.2.4.0+opam.tar.gz dst=/home/guillaume/.opam/repo/coq-repo/archives/coq:flocq.2.4.0+opam.tar.gz

instead of downloading the tarballs described in the repository.

This is probably not a bug, probably just a misconfiguration, but the documentation was not helpful in understanding why rsync is used rather than curl for this local repository.

silene commented 9 years ago

Actually, from looking at opam code, it does look like a real bug. As far as I understand the issue, as soon as opam notices a colon in a file name, even if there are slashes before it, it assumes that it is a remote file that should be fetched by rsync. More precisely, in the OpamLocal.rsync_file function, rather than executing

Done (Not_available src_s)

as it should, it executes

call_rsync ( rsync_arg :: args @ [ src_s; dst_s ])

which leads to the aforementioned failure. Just to be sure, I removed the colons from the coq/repo-stable repository. Then downloading and building succeeded.

AltGr commented 9 years ago

Ok, thanks for the report! Is this on 1.2 or master ? Because I've just been sanitising download handling.

AltGr commented 9 years ago

I can confirm the bug

AltGr commented 9 years ago

Ok, here is the issue: it's expected that OPAM will attempt to get the +opam.tar.gz archive, for http and rsync repos -- this allows for local caching of archive using opam-admin, and local testing of repositories. The issue is that rsync returns the same error on failing to copy a single file's attributes, on which we just want to print a warning, and completely failing, so OPAM doesn't treat it a fatal error.

I'll be adding a test after the call to rsync to check if it was successful.