Voodoo / odoc error reading cmi file.

[tmcgilchrist]

Found when building vchan.6.0.0 Build logs at https://docs.ci.ocaml.org/job/2023-02-14/101944-voodoo-do-24e042

No dune: Missing
Found no other pages
libraries: []
compile command: odoc compile ./p.mld -o compile/page-p.odoc --child
                   page-"ocaml-src"
File "./p.mld", line 2, characters 12-21:
Warning: Unknown reference qualifier 'ocaml'.
compile command: odoc compile ./ocaml-src.mld -o
                   compile/p/page-ocaml-src.odoc --parent "p" -I compile
                   --child page-"4.14.0"
File "./ocaml-src.mld", line 2, characters 17-18:
Warning: Child label is not allowed in the last component of a reference path.
Suggestion: 'child-0' should be first.
compile command: odoc compile ./4.14.0.mld -o
                   compile/p/ocaml-src/page-4.14.0.odoc --parent "ocaml-src"
                   -I compile/p --child page-"doc"
compile command: odoc compile ./doc.mld -o
                   compile/p/ocaml-src/4.14.0/page-doc.odoc --parent "4.14.0"
                   -I compile/p/ocaml-src --child module-b
odoc: internal error, uncaught exception:
      Cmi_format.Error(_)
      Raised at Cmi_format.read_cmi in file "file_formats/cmi_format.ml", line 83, characters 6-21
      Called from Odoc_odoc__Depends.for_compile_step_cmi_or_cmti in file "src/odoc/depends.ml", line 37, characters 18-62
      Called from Dune__exe__Main.Depends.Compile.list_dependencies in file "src/odoc/bin/main.ml", line 689, characters 17-72
      Called from Cmdliner_term.app.(fun) in file "cmdliner_term.ml", line 24, characters 19-24
      Called from Cmdliner_eval.run_parser in file "cmdliner_eval.ml", line 34, characters 37-44
voodoo-do: internal error, uncaught exception:
           Failure("Error: run ['odoc' 'compile-deps'\n     'prep/universes/a1e024d19dfa778717b410c29d2e14e6/ocaml-src/4.14.0/lib/ocaml-src/testsuite/tests/no-alias-deps/b.cmi']: exited with 2")
           Raised at Stdlib.failwith in file "stdlib.ml", line 29, characters 17-33
           Called from Voodoo_lib__Odoc.compile_deps in file "src/voodoo/odoc.ml", line 38, characters 8-94
           Called from Voodoo_lib__Do.get_source_info in file "src/voodoo/do.ml", line 65, characters 12-34
           Called from Stdlib__List.map in file "list.ml", line 92, characters 20-23
           Called from Voodoo_lib__Listm.(>>=) in file "src/voodoo/listm.ml" (inlined), line 7, characters 31-45
           Called from Voodoo_lib__Do.run in file "src/voodoo/do.ml", line 195, characters 12-43
           Called from Cmdliner_term.app.(fun) in file "cmdliner_term.ml", line 24, characters 19-24
           Called from Cmdliner_eval.run_parser in file "cmdliner_eval.ml", line 34, characters 37-44
"/bin/bash" "-c"
"(OCAMLRUNPARAM=b opam exec -- /home/opam/voodoo-do -p ocaml-src -b ) && (shopt -s nullglob && ((tar -cvf compile/p/ocaml-src/4.14.0.tar compile/p/ocaml-src/4.14.0/*  && rm -R compile/p/ocaml-src/4.14.0/* && mv compile/p/ocaml-src/4.14.0.tar compile/p/ocaml-src/4.14.0/content.tar) || (echo 'Empty directory'))) && (mkdir -p linked && mkdir -p epoch-161b629bd52882d49754967e7f795117 && mv linked epoch-161b629bd52882d49754967e7f795117/) && (mkdir -p epoch-161b629bd52882d49754967e7f795117/linked/p/ocaml-src/4.14.0) && (shopt -s nullglob && ((tar -cvf epoch-161b629bd52882d49754967e7f795117/linked/p/ocaml-src/4.14.0.tar epoch-161b629bd52882d49754967e7f795117/linked/p/ocaml-src/4.14.0/*  && rm -R epoch-161b629bd52882d49754967e7f795117/linked/p/ocaml-src/4.14.0/* && mv epoch-161b629bd52882d49754967e7f795117/linked/p/ocaml-src/4.14.0.tar epoch-161b629bd52882d49754967e7f795117/linked/p/ocaml-src/4.14.0/content.tar) || (echo 'Empty directory')))" failed with exit status 125
2023-02-14 10:22.35: Job failed: Failed: Build failed

[sabine]

According to this StackOverflow answer, this could be a mismatch of ocaml compiler versions used in the voodoo-prep job and in the voodoo-do job.

This sounds like a thing ocaml-docs-ci must ensure: that the same ocaml compiler version is used for both steps? :thinking:

[jonludlam]

The problem is occurring while processing the package ocaml-src, which contains a copy of the ocaml source tree. The problem is that there's a cmi file embedded in the source tree in a test directory - and prep has plucked this out and handed it over to voodoo-do for processing. I don't quite know why odoc is failing to read it, but whatever the reason there's nothing good going to come out of it. We've dealt with a similar issue in the past by blacklisting ocaml-secondary-compiler (https://github.com/ocaml-doc/voodoo/blob/main/src/voodoo-prep/main.ml#L20) and we could do a similar thing here - there will never be any use in running odoc on the contents of the ocaml-src!

[novemberkilo]

This appears to have popped up again - now in the context of the Melange team wanting to know why their docs are not available. The issue occurs in this failure https://docs.ci.ocaml.org/job/2023-06-04/162831-voodoo-do-7aa510 and has a similar stack trace to what is posted above.

[jonludlam]

The package blacklisting is probably too blunt an instrument for this. We should either check the cmi is valid in voodoo or just catch the error returned from odoc and remove it from the list.

[sabine]

Indeed, let's check that the cmi is valid during voodoo-prep, so that only valid cmi's (those matching the ocaml compiler version) are part of the output of voodoo-prep