search

ocurrent / opam-repo-ci

An OCurrent pipeline for testing submissions to opam-repository
19 stars 21 forks source link

`OpamPp.Bad_format` #291

Open raphael-proust opened 2 months ago

raphael-proust commented 2 months ago

In the CI analysis phase https://opam.ci.ocaml.org/github/ocaml/opam-repository/commit/dbaea0a12fb0d808ae1a593c296b163ffba4a896/variant/(analysis) there's an opam failure which is uncaught:

2024-04-14 06:51.09: New job: Analyse
2024-04-14 06:51.09: Checking out commit dbaea0a1. To reproduce:
                       git clone --recursive "https://github.com/ocaml/opam-repository.git" && cd "opam-repository" && git fetch origin "refs/pull/25670/head" && git reset --hard dbaea0a1
2024-04-14 06:51.09: Exec: "cp" "-a" "--" "/var/lib/ocurrent/var/git/opam-repository.git-a0bc4b41ddb868605dc1500002c27cfa79faf4389fe53b6754817ea16b48d458/.git" 
                           "/tmp/git-checkout2e67b450"
2024-04-14 06:51.10: Exec: "git" "-C" "/tmp/git-checkout2e67b450" "submodule" 
                           "deinit" "--force" "--all"
2024-04-14 06:51.10: Exec: "git" "-C" "/tmp/git-checkout2e67b450" "reset" 
                           "--hard" "-q" "dbaea0a12fb0d808ae1a593c296b163ffba4a896"
2024-04-14 06:51.14: Exec: "git" "-c" "protocol.file.allow=always" "-C" 
                           "/tmp/git-checkout2e67b450" "submodule" "update" 
                           "--recursive" "--init" "--no-fetch"
2024-04-14 06:51.14: Exec: "git" "merge" "-q" "--" "6c3f73f42890cc19f81eb1dec8023c2cd7b8b5cd"
2024-04-14 06:51.14: Exec: "git" "diff" "--name-only" "6c3f73f42890cc19f81eb1dec8023c2cd7b8b5cd"
2024-04-14 06:51.15: Exec: "git" "show" "6c3f73f42890cc19f81eb1dec8023c2cd7b8b5cd:packages/conf-protoc-dev/conf-protoc-dev.1.0.0/opam"
fatal: path 'packages/conf-protoc-dev/conf-protoc-dev.1.0.0/opam' exists on disk, but not in '6c3f73f42890cc19f81eb1dec8023c2cd7b8b5cd'
2024-04-14 06:51.15: Exec: "git" "show" "6c3f73f42890cc19f81eb1dec8023c2cd7b8b5cd:packages/ocaml-protoc-plugin/ocaml-protoc-plugin.6.0.0/opam"
fatal: path 'packages/ocaml-protoc-plugin/ocaml-protoc-plugin.6.0.0/opam' exists on disk, but not in '6c3f73f42890cc19f81eb1dec8023c2cd7b8b5cd'
2024-04-14 06:51.30: Job failed: OpamPp.Bad_format(_) -- Raised by primitive operation at Lwt_unix.read_bigarray.(fun) in file "src/unix/lwt_unix.cppo.ml", line 688, characters 8-59
Called from Lwt_unix.wrap_syscall.(fun) in file "src/unix/lwt_unix.cppo.ml", line 571, characters 17-28

I'm not sure this is the right place for this report. But I don't know whether it's an issue with opam itself or with the way it is called in by the opam-repo-ci so I'm opening it here for now and we can discuss location if needs be.

mtelvers commented 2 months ago

I think the issue here is the way the exception is being handled by Lwt in Opam-repo-ci. The failure in the job report is OpamPp.Bad_format which correctly indicates that the opam file could not be parsed.

https://github.com/ocurrent/opam-repo-ci/blob/dbf511d310b73c9ceae23f503c023efda49f4a90/lib/analyse.ml#L166-L171

If I test with opam-repo-ci-local I see the correct failwith message

2024-04-15 11:00.41: New job: Analyse
2024-04-15 11:00.41: Checking out commit 07c480d1. To reproduce:
                       git clone --recursive "/home/mtelvers/opam-repository" -b "pr25670" && cd "opam-repository" && git reset --hard 07c480d1
2024-04-15 11:00.41: Exec: "cp" "-a" "--" "/home/mtelvers/opam-repository/.git" 
                           "/tmp/git-checkout3076cd59"
2024-04-15 11:00.41: Exec: "git" "-C" "/tmp/git-checkout3076cd59" "submodule" 
                           "deinit" "--force" "--all"
2024-04-15 11:00.41: Exec: "git" "-C" "/tmp/git-checkout3076cd59" "reset" 
                           "--hard" "-q" "07c480d1680483d259d8e96c422aa02909b4fd1c"
2024-04-15 11:00.45: Exec: "git" "-c" "protocol.file.allow=always" "-C" 
                           "/tmp/git-checkout3076cd59" "submodule" "update" 
                           "--recursive" "--init" "--no-fetch"
2024-04-15 11:00.45: Exec: "git" "merge" "-q" "--" "6c3f73f42890cc19f81eb1dec8023c2cd7b8b5cd"
2024-04-15 11:00.45: Exec: "git" "diff" "--name-only" "6c3f73f42890cc19f81eb1dec8023c2cd7b8b5cd"
2024-04-15 11:00.46: Exec: "git" "show" "6c3f73f42890cc19f81eb1dec8023c2cd7b8b5cd:packages/conf-protoc-dev/conf-protoc-dev.1.0.0/opam"
fatal: path 'packages/conf-protoc-dev/conf-protoc-dev.1.0.0/opam' exists on disk, but not in '6c3f73f42890cc19f81eb1dec8023c2cd7b8b5cd'
2024-04-15 11:00.46: Exec: "git" "show" "6c3f73f42890cc19f81eb1dec8023c2cd7b8b5cd:packages/ocaml-protoc-plugin/ocaml-protoc-plugin.6.0.0/opam"
fatal: path 'packages/ocaml-protoc-plugin/ocaml-protoc-plugin.6.0.0/opam' exists on disk, but not in '6c3f73f42890cc19f81eb1dec8023c2cd7b8b5cd'
2024-04-15 11:00.58: Job failed: At packages/ocaml-protoc-plugin/ocaml-protoc-plugin.6.0.0/opam:29:16-29:17::
'.' is not a valid token

However, testing the same commit using the opam-repo-ci-service does not give the error message.

2024-04-15 10:37.18: New job: Analyse
2024-04-15 10:37.18: Checking out commit 07c480d1. To reproduce:
                       git clone --recursive "https://github.com/mtelvers/opam-repository.git" && cd "opam-repository" && git fetch origin "refs/pull/11/head" && git reset --hard 07c480d1
2024-04-15 10:37.18: Exec: "cp" "-a" "--" "/home/mtelvers/opam-repo-ci/var/git/opam-repository.git-780a482f06f13c0e99a5fd9a91eca9badbd8accdecc0edc0362b986355279f43/.git" 
                           "/tmp/git-checkout14e0426c"
2024-04-15 10:37.19: Exec: "git" "-C" "/tmp/git-checkout14e0426c" "submodule" 
                           "deinit" "--force" "--all"
2024-04-15 10:37.19: Exec: "git" "-C" "/tmp/git-checkout14e0426c" "reset" 
                           "--hard" "-q" "07c480d1680483d259d8e96c422aa02909b4fd1c"
2024-04-15 10:37.23: Exec: "git" "-c" "protocol.file.allow=always" "-C" 
                           "/tmp/git-checkout14e0426c" "submodule" "update" 
                           "--recursive" "--init" "--no-fetch"
2024-04-15 10:37.23: Exec: "git" "merge" "-q" "--" "6c3f73f42890cc19f81eb1dec8023c2cd7b8b5cd"
2024-04-15 10:37.23: Exec: "git" "diff" "--name-only" "6c3f73f42890cc19f81eb1dec8023c2cd7b8b5cd"
2024-04-15 10:37.23: Exec: "git" "show" "6c3f73f42890cc19f81eb1dec8023c2cd7b8b5cd:packages/conf-protoc-dev/conf-protoc-dev.1.0.0/opam"
fatal: path 'packages/conf-protoc-dev/conf-protoc-dev.1.0.0/opam' exists on disk, but not in '6c3f73f42890cc19f81eb1dec8023c2cd7b8b5cd'
2024-04-15 10:37.23: Exec: "git" "show" "6c3f73f42890cc19f81eb1dec8023c2cd7b8b5cd:packages/ocaml-protoc-plugin/ocaml-protoc-plugin.6.0.0/opam"
fatal: path 'packages/ocaml-protoc-plugin/ocaml-protoc-plugin.6.0.0/opam' exists on disk, but not in '6c3f73f42890cc19f81eb1dec8023c2cd7b8b5cd'
2024-04-15 10:37.35: Job failed: OpamPp.Bad_format(_) -- Raised at Stdlib__String.index_rec in file "string.ml", line 128, characters 19-34
Called from Ansi__Char_stream.find in file "lib/char_stream.ml", line 20, characters 18-46