Opam archive unable to update

[abhi18av]

Hi opam team

I'm trying to install the latest version of eliom and family. I feel for that it's important to be able to run opam update and that throws errors

Abhinavs-MacBook-Pro:~ eklavya$ opam update -vv

<><> Updating package repositories ><><><><><><><><><><><><><><><><><><><><>  🐫
+ /usr/local/bin/wget "--content-disposition" "-t" "3" "-O" "/private/var/folders/gz/sdxvfmgd0_38g225sldvm7j00000gn/T/opam-37384-5a61c5/index.tar.gz.part" "https://opam.ocaml.org/index.tar.gz" "-U" "opam/2.0.0"
- --2018-12-08 19:55:33--  https://opam.ocaml.org/index.tar.gz
- Resolving opam.ocaml.org (opam.ocaml.org)... 192.237.250.17, 2001:4801:7823:76:bb74:1f8e:ff10:627
- Connecting to opam.ocaml.org (opam.ocaml.org)|192.237.250.17|:443... connected.
- ERROR: cannot verify opam.ocaml.org's certificate, issued by ‘CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US’:
-   Unable to locally verify the issuer's authority.
- To connect to opam.ocaml.org insecurely, use `--no-check-certificate'.
[ERROR] Could not update repository "default": OpamDownload.Download_fail(_, "Download command failed: \"/usr/local/bin/wget
        --content-disposition -t 3 -O /private/var/folders/gz/sdxvfmgd0_38g225sldvm7j00000gn/T/opam-37384-5a61c5/index.tar.gz.part
        https://opam.ocaml.org/index.tar.gz -U opam/2.0.0\" exited with code 5 \"ERROR: cannot verify opam.ocaml.org's
        certificate, issued by \226\128\152CN=Let's Encrypt Authority X3,O=Let's Encrypt,C=US\226\128\153:\"")
'opam update -vv' failed.

• Here's my wget version

Abhinavs-MacBook-Pro:~ eklavya$ wget --version
GNU Wget 1.20 built on darwin17.7.0.

-cares +digest -gpgme +https +ipv6 +iri +large-file -metalink +nls
+ntlm +opie -psl +ssl/openssl

Wgetrc:
    /usr/local/etc/wgetrc (system)
Locale:
    /usr/local/Cellar/wget/1.20/share/locale
Compile:
    clang -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/usr/local/etc/wgetrc"
    -DLOCALEDIR="/usr/local/Cellar/wget/1.20/share/locale" -I. -I../lib
    -I../lib -I/usr/local/opt/openssl/include -I/usr/local/include
    -DNDEBUG
Link:
    clang -I/usr/local/include -DNDEBUG -lidn2
    -L/usr/local/opt/openssl/lib -lssl -lcrypto -ldl -lz ftp-opie.o
    openssl.o http-ntlm.o ../lib/libgnu.a -liconv -lintl -Wl,-framework
    -Wl,CoreFoundation -lunistring

Copyright (C) 2015 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
<http://www.gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Originally written by Hrvoje Niksic <hniksic@xemacs.org>.
Please send bug reports and questions to <bug-wget@gnu.org>.

[abhi18av]

Also, tried again after installing gpatch as suggested here https://github.com/ocaml/opam/issues/3639

Abhinavs-MacBook-Pro:~ eklavya$ brew install gpatch
Updating Homebrew...

==> Auto-updated Homebrew!
Updated 3 taps (homebrew/cask-versions, homebrew/core and homebrew/cask).
==> Updated Formulae
eslint                                      osrm-backend                                shfmt

==> Downloading https://homebrew.bintray.com/bottles/gpatch-2.7.6.high_sierra.bottle.tar.gz
######################################################################## 100.0%
==> Pouring gpatch-2.7.6.high_sierra.bottle.tar.gz
🍺  /usr/local/Cellar/gpatch/2.7.6: 10 files, 324.3KB
Abhinavs-MacBook-Pro:~ eklavya$
Abhinavs-MacBook-Pro:~ eklavya$ opam update

[abhi18av]

The problem persists even after upgrading to opam-2.0.1

[pmetzger]

I think the issue is your wget doesn't have access to the Let's Encrypt CA certificates. I'd build a recent version of wget from brew or macports.

[abhi18av]

Hi @pmetzger could you guide me how to do that?

I already installed wget from brew on macOS.

[abhi18av]

An update, running in sudo has worked.

Abhinavs-MacBook-Pro:~ eklavya$ sudo opam update
Password:
[WARNING] Running as root is not recommended

<><> Updating package repositories ><><><><><><><><><><><><><><><><><><><><>  🐫
Processing  1/1: [default: http]
[default] synchronised from https://opam.ocaml.org
Now run 'opam upgrade' to apply any package updates.

[pmetzger]

Running opam as the superuser is very much not recommended.

As for using brew or macports, I'd google for both projects and see which one you like using.

[abhi18av]

Hi @pmetzger I have realized the issue now.

Initially I had installed opam via homebrew which resulted in opam not being able to update or install anything.

Then eventually, after trying a lot many things, I just did the vanilla shell script based installation as mentioned in the website and then everything started working fine.

I've since been able to install and setup 4.07.1+BER without any issues 👍

[chrisfcarroll]

brew upgrade wget -- from 1.20.3 to 1.20.3_2 -- fixed this for me on macos Catalina

[liyishuai]

Failing with wget 1.21.3 from Homebrew, using pre-compiled OPAM 2.1.4 on x86_64 macOS 13.3.1 (a) (22E772610a).

LANG=C opam update -vv

<><> Updating package repositories ><><><><><><><><><><><><><><><><><><><><><><>
+ /usr/bin/tar "xfz" "/Users/liyishuai.lys/.opam/repo/coq-released.tar.gz" "-C" "/private/var/folders/c4/rps468712vz45b73qrcnfdkr0000gq/T/opam-36791-452256"
+ /usr/bin/tar "xfz" "/Users/liyishuai.lys/.opam/repo/default.tar.gz" "-C" "/private/var/folders/c4/rps468712vz45b73qrcnfdkr0000gq/T/opam-36791-452256"
- --2023-05-09 10:06:22--  https://coq.inria.fr/opam/released/index.tar.gz
- Resolving coq.inria.fr (coq.inria.fr)... 51.91.56.51, 2001:41d0:305:2100::8b43
- Connecting to coq.inria.fr (coq.inria.fr)|51.91.56.51|:443... connected.
- ERROR: cannot verify coq.inria.fr's certificate, issued by ‘CN=R3,O=Let's Encrypt,C=US’:
-   Unable to locally verify the issuer's authority.
- To connect to coq.inria.fr insecurely, use `--no-check-certificate'.
[ERROR] Could not update repository "coq-released": OpamDownload.Download_fail(_, "Download command failed: \"/usr/local/bin/wget --content-disposition -t 3 -O
        /private/var/folders/c4/rps468712vz45b73qrcnfdkr0000gq/T/opam-36791-b4f928/index.tar.gz.part -U opam/2.1.4 -- https://coq.inria.fr/opam/released/index.tar.gz\" exited with code 5 \"ERROR: cannot verify
        coq.inria.fr's certificate, issued by \226\128\152CN=R3,O=Let's Encrypt,C=US\226\128\153:\"")
+ /usr/local/bin/wget "--content-disposition" "-t" "3" "-O" "/private/var/folders/c4/rps468712vz45b73qrcnfdkr0000gq/T/opam-36791-493d78/index.tar.gz.part" "-U" "opam/2.1.4" "--" "https://opam.ocaml.org/index.tar.gz"
- --2023-05-09 10:06:45--  https://opam.ocaml.org/index.tar.gz
- Resolving opam.ocaml.org (opam.ocaml.org)... 151.115.76.159, 51.158.232.133, 2001:bc8:1d80:4600::1, ...
- Connecting to opam.ocaml.org (opam.ocaml.org)|151.115.76.159|:443... connected.
- ERROR: cannot verify opam.ocaml.org's certificate, issued by ‘CN=R3,O=Let's Encrypt,C=US’:
-   Unable to locally verify the issuer's authority.
- To connect to opam.ocaml.org insecurely, use `--no-check-certificate'.
[ERROR] Could not update repository "default": OpamDownload.Download_fail(_, "Download command failed: \"/usr/local/bin/wget --content-disposition -t 3 -O
        /private/var/folders/c4/rps468712vz45b73qrcnfdkr0000gq/T/opam-36791-493d78/index.tar.gz.part -U opam/2.1.4 -- https://opam.ocaml.org/index.tar.gz\" exited with code 5 \"ERROR: cannot verify opam.ocaml.org's
        certificate, issued by \226\128\152CN=R3,O=Let's Encrypt,C=US\226\128\153:\"")
'opam update -vv' failed.

LANG=C wget --version
GNU Wget 1.21.3 built on darwin22.1.0.

-cares +digest -gpgme +https +ipv6 +iri +large-file -metalink +nls 
+ntlm +opie -psl +ssl/openssl 

Wgetrc: 
    /usr/local/etc/wgetrc (system)
Locale: 
    /usr/local/Cellar/wget/1.21.3_1/share/locale 
Compile: 
    clang -DHAVE_CONFIG_H -DSYSTEM_WGETRC="/usr/local/etc/wgetrc" 
    -DLOCALEDIR="/usr/local/Cellar/wget/1.21.3_1/share/locale" -I. 
    -I../lib -I../lib -I/usr/local/opt/openssl@3/include 
    -I/usr/local/Cellar/libidn2/2.3.4_1/include -DNDEBUG -g -O2 
Link: 
    clang -I/usr/local/Cellar/libidn2/2.3.4_1/include -DNDEBUG -g -O2 
    -L/usr/local/Cellar/libidn2/2.3.4_1/lib -lidn2 
    -L/usr/local/opt/openssl@3/lib -lssl -lcrypto -ldl -lz 
    ../lib/libgnu.a -liconv -lintl -Wl,-framework -Wl,CoreFoundation 
    -lunistring

The OPAM binary was downloaded from GitHub, as the install script complains that:

bash -c "sh <(curl -fsSL https://raw.githubusercontent.com/ocaml/opam/master/shell/install.sh)"

## Downloading opam 2.1.4 for macos on x86_64...
There may not yet be a binary release for your architecture or OS, sorry.
See https://github.com/ocaml/opam/releases/tag/2.1.4 for pre-compiled binaries,
or run 'make cold' from https://github.com/ocaml/opam/archive/2.1.4.tar.gz
to build from scratch

[pmetzger]

@liyishuai This is almost always a certificate issue with the provider of your wget binary and not anything to do with opam as such.

[liyishuai]

Solution to my case above:

echo 'check_certificate=off' >> $HOME/.wgetrc

Inspired by https://github.com/ocaml/opam/issues/4857#issuecomment-933833172

[pmetzger]

That leaves you vulnerable though. It's not the right solution.

[bnse]

❯ opam update

<><> Updating package repositories ><><><><><><><><><><><><><><><><><><><><> 🐫 [ERROR] Could not update repository "default": OpamDownload.Downloadfail(, "Download command failed: \"/opt/homebrew/bin/wget --content-disposition -t 3 -O /private/var/folders/3w/4pzvbsl911d9h1h4y1yhgxnr0000gn/T/opam-42314-1bf6f4/index.tar.gz.part -U opam/2.1.5 -- https://opam.ocaml.org/index.tar.gz\" exited with code 5 \"ERROR: cannot verify opam.ocaml.org's certificate, issued by \226\128\152CN=R3,O=Let's Encrypt,C=US\226\128\153:\"")

1. install the ca-certificates $ brew install ca-certificates

$ brew unlink ca-certificates --dry-run Would remove: /opt/homebrew/share/ca-certificates

$ file /opt/homebrew/share/ca-certificates/cacert.pem /opt/homebrew/share/ca-certificates/cacert.pem: Unicode text, UTF-8 text

2. set wget use the ca_file $ cat ~/.wgetrc ca_certificate=/opt/homebrew/share/ca-certificates/cacert.pem

$ opam update && opam upgrade done.