search

ocurrent / ocaml-ci

A CI for OCaml projects
https://ocaml.ci.dev
111 stars 75 forks source link

`lint-doc` job failing on installation of `conf-git` #934

Closed shonfeder closed 3 months ago

shonfeder commented 3 months ago

Context

As reported at https://github.com/ocurrent/ocaml-ci/pull/932#issuecomment-2118195395, the ocaml-ci lint-doc job is failing due to (what appears to be) ascii escape sequences being included in the package name when trying to install conf-git:

``` /src: (run (cache (opam-archives (target /home/opam/.opam/download-cache))) (network host) (shell "opam depext -i conf-m4 && opam depext -i dune 'odoc>=1.5.0'")) Opam plugin "depext" may require upgrading/reinstalling. Reinstall the plugin on the current switch? [Y/n] y [WARNING] Opam package conf-git.1.1 depends on the following system package that can no longer be found: git The following actions will be performed: - recompile opam-depext 1.2.1-1 <><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><> -> retrieved opam-depext.1.2.1-1 (cached) -> removed opam-depext.1.2.1-1 -> installed opam-depext.1.2.1-1 Done. <><> opam-depext.1.2.1-1 installed successfully <><><><><><><><><><><><><><><><> => opam-depext is unnecessary when used with opam >= 2.1. Please use opam install directly instead # Run eval $(opam env) to update the current shell environment <><> Carrying on to "opam depext -i conf-m4" ><><><><><><><><><><><><><><><><><> You are using opam 2.1+, where external dependency handling has been integrated: consider calling opam directly, the 'depext' plugin interface is provided for backwards compatibility only # Detecting depexts using vars: arch=x86_64, os=linux, os-distribution=alpine, os-family=alpine [WARNING] Opam package conf-git.1.1 depends on the following system package that can no longer be found: git # The following system packages are needed: m4 [WARNING] Opam package conf-git.1.1 depends on the following system package that can no longer be found: git opam: PACKAGES... arguments: Invalid character in package name "\027[01;04mconf-git\027[0m\027[04m" Usage: opam install [OPTION]... [PACKAGES]... Try `opam install --help' or `opam --help' for more information. "/usr/bin/env" "bash" "-c" "opam depext -i conf-m4 && opam depext -i dune 'odoc>=1.5.0'" failed with exit status 2 2024-05-17 15:44.40: Job failed: Failed: Build failed ```

Step(s) to reproduce

Either, rerun the failed lint-doc jobs on the master branch here or, execute this shell script:

```sh #!/usr/bin/env sh (git clone --recursive "https://github.com/ocurrent/ocaml-ci.git" || true) && cd "ocaml-ci" && git fetch origin "refs/pull/932/head" && git reset --hard 1a9c144d cat > Dockerfile <<'END-OF-DOCKERFILE' FROM ocaml/opam@sha256:77e8da3284390873fa4e51e9c10a7377d6fa0524845bb813c99bf3b328fa84c3 # alpine-3.19-4.14_opam-2.1 USER 1000:1000 ENV CLICOLOR_FORCE="1" ENV OPAMCOLOR="always" WORKDIR /src RUN sudo ln -f /usr/bin/opam-2.1 /usr/bin/opam RUN opam init --reinit -ni RUN uname -rs && opam exec -- ocaml -version && opam --version WORKDIR /src RUN sudo chown opam /src RUN cd ~/opam-repository && (git cat-file -e 8907d75af817bb5596ec76be769cd0451b41a326 || git fetch origin master) && git reset -q --hard 8907d75af817bb5596ec76be769cd0451b41a326 && git log --no-decorate -n1 --oneline && opam update -u RUN mkdir -p 'solver-service/' 'ocurrent/' 'ocluster/' 'ocluster/obuilder/' 'ocluster/' 'ocaml-version/' 'ocaml-dockerfile/' COPY --chown=1000:1000 solver-service/solver-worker.opam solver-service/solver-service.opam solver-service/solver-service-api.opam solver-service/ COPY --chown=1000:1000 ocurrent/current_web.opam ocurrent/current_ssh.opam ocurrent/current_slack.opam ocurrent/current_rpc.opam ocurrent/current_gitlab.opam ocurrent/current_github.opam ocurrent/current_git.opam ocurrent/current_examples.opam ocurrent/current_docker.opam ocurrent/current.opam ocurrent/ COPY --chown=1000:1000 ocluster/ocluster.opam ocluster/ocluster-worker.opam ocluster/ocluster-api.opam ocluster/ COPY --chown=1000:1000 ocluster/obuilder/obuilder.opam ocluster/obuilder/obuilder-spec.opam ocluster/obuilder/ COPY --chown=1000:1000 ocluster/current_ocluster.opam ocluster/ COPY --chown=1000:1000 ocaml-version/ocaml-version.opam ocaml-version/ COPY --chown=1000:1000 ocaml-dockerfile/dockerfile.opam ocaml-dockerfile/dockerfile-opam.opam ocaml-dockerfile/dockerfile-cmd.opam ocaml-dockerfile/ COPY --chown=1000:1000 ocaml-ci.opam ocaml-ci-web.opam ocaml-ci-service.opam ocaml-ci-gitlab.opam ocaml-ci-client.opam ocaml-ci-api.opam ./ RUN opam pin add -yn solver-worker.dev 'solver-service/' && \ opam pin add -yn solver-service.dev 'solver-service/' && \ opam pin add -yn solver-service-api.dev 'solver-service/' && \ opam pin add -yn current_web.dev 'ocurrent/' && \ opam pin add -yn current_ssh.dev 'ocurrent/' && \ opam pin add -yn current_slack.dev 'ocurrent/' && \ opam pin add -yn current_rpc.dev 'ocurrent/' && \ opam pin add -yn current_gitlab.dev 'ocurrent/' && \ opam pin add -yn current_github.dev 'ocurrent/' && \ opam pin add -yn current_git.dev 'ocurrent/' && \ opam pin add -yn current_examples.dev 'ocurrent/' && \ opam pin add -yn current_docker.dev 'ocurrent/' && \ opam pin add -yn current.dev 'ocurrent/' && \ opam pin add -yn ocluster.dev 'ocluster/' && \ opam pin add -yn ocluster-worker.dev 'ocluster/' && \ opam pin add -yn ocluster-api.dev 'ocluster/' && \ opam pin add -yn obuilder.dev 'ocluster/obuilder/' && \ opam pin add -yn obuilder-spec.dev 'ocluster/obuilder/' && \ opam pin add -yn current_ocluster.dev 'ocluster/' && \ opam pin add -yn ocaml-version.dev 'ocaml-version/' && \ opam pin add -yn dockerfile.dev 'ocaml-dockerfile/' && \ opam pin add -yn dockerfile-opam.dev 'ocaml-dockerfile/' && \ opam pin add -yn dockerfile-cmd.dev 'ocaml-dockerfile/' && \ opam pin add -yn ocaml-ci.dev './' && \ opam pin add -yn ocaml-ci-web.dev './' && \ opam pin add -yn ocaml-ci-service.dev './' && \ opam pin add -yn ocaml-ci-gitlab.dev './' && \ opam pin add -yn ocaml-ci-client.dev './' && \ opam pin add -yn ocaml-ci-api.dev './' ENV DEPS="0install-solver.2.18 alcotest.1.7.0 alcotest-lwt.1.7.0 angstrom.0.16.0 ansi.0.7.0 arp.3.1.1 asetmap.0.8.1 asn1-combinators.0.2.6 astring.0.8.5 atd.2.15.0 atdgen.2.15.0 atdgen-runtime.2.15.0 awa.0.3.0 awa-mirage.0.3.0 base.v0.16.3 base-bigarray.base base-bytes.base base-threads.base base-unix.base base64.3.5.1 bigarray-compat.1.1.0 bigarray-overlap.0.2.1 bigstringaf.0.9.1 biniou.1.2.2 bos.0.2.1 ca-certs.0.2.3 ca-certs-nss.3.98 camlp-streams.5.0.1 capnp.3.6.0 capnp-rpc.1.2.3 capnp-rpc-lwt.1.2.3 capnp-rpc-net.1.2.3 capnp-rpc-unix.1.2.3 caqti.1.9.0 caqti-lwt.1.9.0 carton.0.7.1 carton-git.0.7.1 carton-lwt.0.7.1 cf.0.5.0 cf-lwt.0.5.0 checkseum.0.5.2 cmdliner.1.2.0 cohttp.6.0.0~alpha2 cohttp-lwt.6.0.0~alpha2 cohttp-lwt-unix.6.0.0~alpha2 conduit.6.2.2 conduit-lwt.6.2.2 conduit-lwt-unix.6.2.2 conf-bash.1 conf-capnproto.2 conf-git.1.1 conf-gmp.4 conf-gmp-powm-sec.3 conf-graphviz.0.1 conf-libev.4-12 conf-libffi.2.0.0 conf-libssl.4 conf-pkg-config.3 conf-sqlite3.1 conf-which.1 cppo.1.6.9 crunch.3.3.1 csexp.1.5.2 cstruct.6.2.0 cstruct-lwt.6.2.0 cstruct-unix.6.2.0 csv.2.4 ctypes.0.22.0 ctypes-foreign.0.22.0 current.dev current_docker.dev current_git.dev current_github.dev current_gitlab.dev current_incr.0.6.1 current_ocluster.dev current_rpc.dev current_web.dev decompress.1.5.3 digestif.1.2.0 dns.7.0.3 dns-client.7.0.3 dns-client-lwt.7.0.3 dns-client-mirage.7.0.3 dockerfile.dev dockerfile-opam.dev domain-name.0.4.0 dream.1.0.0~alpha5 dream-httpaf.1.0.0~alpha2 dream-pure.1.0.0~alpha2 duff.0.5 dune.3.15.2 dune-build-info.3.15.2 dune-configurator.3.15.2 duration.0.2.1 easy-format.1.3.4 emile.1.1 encore.0.8 eqaf.0.9 ethernet.3.2.0 extunix.0.4.1 faraday.0.8.2 faraday-lwt.0.8.2 faraday-lwt-unix.0.8.2 fmt.0.9.0 fpath.0.7.3 fsevents.0.3.0 fsevents-lwt.0.3.0 git.3.15.0 git-mirage.3.15.0 git-paf.3.15.0 git-unix.3.15.0 github.4.4.1 github-data.4.4.1 github-unix.4.4.1 gitlab.0.1.8 gitlab-unix.0.1.8 gmap.0.3.0 graphql.0.14.0 graphql-lwt.0.14.0 graphql_parser.0.14.0 h2.0.11.0 happy-eyeballs.0.6.0 happy-eyeballs-lwt.0.6.0 happy-eyeballs-mirage.0.6.0 hex.1.5.0 hkdf.1.0.4 hmap.0.8.1 hpack.0.11.0 http.6.0.0~alpha2 httpaf.0.7.1 hxd.0.3.2 ISO8601.0.2.6 inotify.2.5 integers.0.7.0 ipaddr.5.5.0 ipaddr-cstruct.5.5.0 ipaddr-sexp.5.5.0 irmin-watcher.0.5.0 ke.0.6 logs.0.7.0 lru.0.3.1 lwt.5.7.0 lwt-dllist.1.0.1 lwt_ppx.2.1.0 lwt_ssl.1.2.0 macaddr.5.5.0 macaddr-cstruct.5.5.0 magic-mime.1.3.1 menhir.20231231 menhirCST.20231231 menhirLib.20231231 menhirSdk.20231231 metrics.0.4.1 mimic.0.0.6 mimic-happy-eyeballs.0.0.6 mirage-clock.4.2.0 mirage-clock-unix.4.2.0 mirage-crypto.0.11.3 mirage-crypto-ec.0.11.3 mirage-crypto-pk.0.11.3 mirage-crypto-rng.0.11.3 mirage-crypto-rng-lwt.0.11.3 mirage-flow.3.0.0 mirage-kv.6.1.1 mirage-net.4.0.0 mirage-random.3.0.0 mirage-runtime.4.5.1 mirage-time.3.0.0 mirage-unix.5.0.1 mtime.2.0.0 multipart_form.0.6.0 multipart_form-lwt.0.6.0 num.1.5 obuilder-spec.dev ocaml.4.14.2 ocaml-base-compiler.4.14.2 ocaml-compiler-libs.v0.12.4 ocaml-config.2 ocaml-options-vanilla.1 ocaml-syntax-shims.1.0.0 ocaml-version.dev ocamlbuild.0.14.3 ocamlfind.1.9.6 ocamlgraph.2.1.0 ocluster-api.dev ocplib-endian.1.2 omigrate.0.3.2 opam-0install.0.4.3 opam-core.2.1.6 opam-file-format.2.1.6 opam-format.2.1.6 opam-repository.2.1.6 opam-state.2.1.6 optint.0.3.0 paf.0.5.0 parsexp.v0.16.0 pbkdf.1.2.0 pecu.0.7 pgx.2.2 pgx_lwt.2.2 pgx_lwt_unix.2.2 ppx_compare.v0.16.0 ppx_custom_printf.v0.16.0 ppx_derivers.1.2.1 ppx_deriving.5.2.1 ppx_deriving_yojson.3.7.0 ppx_sexp_conv.v0.16.0 ppxlib.0.32.1 prettym.0.0.3 prometheus.1.2 prometheus-app.1.2 psq.0.2.1 ptime.1.1.0 randomconv.0.1.3 re.1.11.0 res.5.0.1 result.1.5 routes.2.0.0 rresult.0.7.0 seq.base session.0.5.0 session-cohttp.0.5.0 session-cohttp-lwt.0.5.0 sexplib.v0.16.0 sexplib0.v0.16.0 solver-service.dev solver-service-api.dev sqlite3.5.1.0 ssl.0.7.0 stdint.0.7.2 stdio.v0.16.0 stdlib-shims.0.3.0 stringext.1.6.0 tailwindcss.dev tar.2.6.0 tar-unix.2.6.0 tcpip.8.0.0 timedesc.3.1.0 timedesc-tzdb.3.1.0 timedesc-tzlocal.3.1.0 tls.0.17.3 tls-lwt.0.17.3 tls-mirage.0.17.3 topkg.1.0.7 tyxml.4.6.0 unstrctrd.0.4 uri.4.4.0 uri-sexp.4.4.0 uuidm.0.9.8 uutf.1.0.3 x509.0.16.5 yojson.2.1.2 zarith.1.13" ENV CI="true" ENV OCAMLCI="true" RUN opam update --depexts && opam install --cli=2.1 --depext-only -y ocaml-ci.dev ocaml-ci-web.dev ocaml-ci-service.dev ocaml-ci-gitlab.dev ocaml-ci-client.dev ocaml-ci-api.dev $DEPS RUN opam install $DEPS RUN opam depext -i conf-m4 && opam depext -i dune 'odoc>=1.5.0' COPY --chown=1000:1000 . /src/ RUN ODOC_WARN_ERROR=false opam exec -- dune build @doc || (echo "dune build @doc failed"; exit 2) END-OF-DOCKERFILE docker build . ```

Technical details

Making the following small fix to the script lets the depext installation go thru:

@|-59,3 +59,3 ============================================================
 |RUN opam install $DEPS
-|RUN opam depext -i conf-m4 && opam depext -i dune 'odoc>=1.5.0'
+|RUN opam install -y conf-m4 && opam install -y dune 'odoc>=1.5.0'
 |COPY --chown=1000:1000 . /src/

This also avoids the warning resulting from use of the deprecated depext plugin.

shonfeder commented 3 months ago

In fact, as per https://github.com/ocurrent/ocaml-ci/pull/258, the installation of conf-m4 shouldn't be needed at all now either.