opam on windows: Error while running ocamlc -version during opam install

[JonasOberhauser]

Trying to install coq-core.8.19.2

#=== ERROR while compiling coq-core.8.19.2 ====================================#
# context     2.2.0 | win32/x86_64 | ocaml.5.2.0 | https://opam.ocaml.org#2b34893577f5b57ae4183fd6260e7a3ef7499268
# path        ~\AppData\Local\opam\default\.opam-switch\build\coq-core.8.19.2
# command     ~\AppData\Local\opam\default\.opam-switch\build\coq-core.8.19.2\./configure -prefix D:\Users\...\AppData\Local\opam\default -mandir D:\Users\...\AppData\Local\opam\default\man -libdir D:\Users\...\AppData\Local\opam\default\lib/coq -native-compiler no
# exit-code   1
# env-file    ~\AppData\Local\opam\log\coq-core-40224-fa2d91.env
# output-file ~\AppData\Local\opam\log\coq-core-40224-fa2d91.out
### output ###
# Uncaught exception: Invalid_argument("index out of bounds")
# Error while running 'D:\Users\...\AppData\Local\opam\default\bin/ocamlfind.exe ocamlc -version' (exit code 3)
# Configuration script failed!

Manually running D:\Users\...\AppData\Local\opam\default\bin/ocamlfind.exe ocamlc -version outputs 5.2.0

[kit-ty-kate]

Could you try re-running it with CAMLRUNPARAM=b exported in your environment so we can see the backtrace of the Invalid_argument exception?

[hhugo]

I wonder if it could be https://github.com/ocaml/ocamlfind/pull/64

[JonasOberhauser]

Could you try re-running it with CAMLRUNPARAM=b exported in your environment so we can see the backtrace of the Invalid_argument exception?

What am I doing wrong?

...@___ MINGW64 ~
$ export CAMLRUNPARAM=b

...@___ MINGW64 ~
$ echo $CAMLRUNPARAM
b

...@___ MINGW64 ~
$ opam install coq-core
The following actions will be performed:
=== install 1 package
  - install coq-core 8.19.2

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
-> retrieved coq-core.8.19.2  (cached)
[ERROR] The compilation of coq-core.8.19.2 failed at "./configure -prefix D:\\Users\\...\\AppData\\Local\\opam\\default -mandir D:\\Users\\...\\AppData\\Local\\opam\\default\\man -libdir D:\\Users\\...\\AppData\\Local\\opam\\default\\lib/coq -native-compiler no".

#=== ERROR while compiling coq-core.8.19.2 ====================================#
# context     2.2.0 | win32/x86_64 | ocaml.5.2.0 | https://opam.ocaml.org#2b34893577f5b57ae4183fd6260e7a3ef7499268
# path        ~\AppData\Local\opam\default\.opam-switch\build\coq-core.8.19.2
# command     ~\AppData\Local\opam\default\.opam-switch\build\coq-core.8.19.2\./configure -prefix D:\Users\...\AppData\Local\opam\default -mandir D:\Users\...\AppData\Local\opam\default\man -libdir D:\Users\...\AppData\Local\opam\default\lib/coq -native-compiler no
# exit-code   1
# env-file    ~\AppData\Local\opam\log\coq-core-14964-4e7eac.env
# output-file ~\AppData\Local\opam\log\coq-core-14964-4e7eac.out
### output ###
# Uncaught exception: Invalid_argument("index out of bounds")
# Error while running 'D:\Users\...\AppData\Local\opam\default\bin/ocamlfind.exe ocamlc -version' (exit code 3)
# Configuration script failed!

<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
+- The following actions failed
| - build coq-core 8.19.2
+-
- No changes have been performed
# Run eval $(opam env) to update the current shell environment

( ~/AppData/Local/opam/log/coq-core-14964-4e7eac.out also has no additional information)

Same results with SET CAMLRUNPARAM=b in my powershell

[JonasOberhauser]

I wonder if it could be ocaml/ocamlfind#64

Good question, how could I test that?

[kit-ty-kate]

mmh, weird. Maybe coq's configure script scrubs the environment somehow or something else. I'm not sure how to fix that.

Good question, how could I test that?

opam pin add https://github.com/db4/ocamlfind.git#win32-fix

should do it i believe

[JonasOberhauser]

mmh, weird. Maybe coq's configure script scrubs the environment somehow or something else. I'm not sure how to fix that.

Good question, how could I test that?

opam pin add https://github.com/db4/ocamlfind.git#win32-fix

should do it i believe

...@___ MINGW64 /
$ opam pin add https://github.com/db4/ocamlfind.git#win32-fix
ocamlfind is now pinned to git+https://github.com/db4/ocamlfind.git#win32-fix (version 1.9.6.git)

The following actions will be performed:
=== recompile 3 packages
  - recompile ppx_deriving 6.0.2                       [uses ocamlfind]
  - recompile sel          0.4.0                       [uses ppx_deriving]
  - recompile zarith       1.14                        [uses ocamlfind]
=== upgrade 1 package
  - upgrade   ocamlfind    1.9.5 to 1.9.6.git (pinned)

Proceed with 3 recompilations and 1 upgrade? [y/n] y

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
-> retrieved ppx_deriving.6.0.2  (cached)
-> retrieved sel.0.4.0  (cached)
-> retrieved zarith.1.14  (cached)
[ERROR] The compilation of ocamlfind.1.9.6.git failed at "make all".

#=== ERROR while compiling ocamlfind.1.9.6.git ================================#
# context     2.2.0 | win32/x86_64 | ocaml.5.2.0 | pinned(git+https://github.com/db4/ocamlfind.git#win32-fix#3ce72120ac1ed50025d13e2366ff18781b8d350d)
# path        ~\AppData\Local\opam\default\.opam-switch\build\ocamlfind.1.9.6.git
# command     ~\AppData\Local\opam\.cygwin\root\bin\make.exe all
# exit-code   2
# env-file    ~\AppData\Local\opam\log\ocamlfind-41844-ce1932.env
# output-file ~\AppData\Local\opam\log\ocamlfind-41844-ce1932.out
### output ###
# [...]
# make[1]: Leaving directory '/cygdrive/d/Users/.../AppData/Local/opam/default/.opam-switch/build/ocamlfind.1.9.6.git/src/findlib'
# /usr/bin/make all-config
# make[1]: Entering directory '/cygdrive/d/Users/.../AppData/Local/opam/default/.opam-switch/build/ocamlfind.1.9.6.git'
# USE_CYGPATH="1"; \
# export USE_CYGPATH; \
# cat findlib.conf.in | \
#      tools/patch '@SITELIB@' 'D:\Users\...\AppData\Local\opam\default\lib' | \
#                tools/patch '@FINDLIB_PATH@' 'D:\Users\...\AppData\Local\opam\default\lib' -p >findlib.conf
# sed: -e expression #1, char 20: unknown option to `s'
# make[1]: *** [Makefile:54: findlib.conf] Error 1
# make[1]: Leaving directory '/cygdrive/d/Users/.../AppData/Local/opam/default/.opam-switch/build/ocamlfind.1.9.6.git'
# make: *** [Makefile:15: all] Error 2

<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><><><>
+- The following actions failed
| - build ocamlfind 1.9.6.git
+-
- No changes have been performed
# Run eval $(opam env) to update the current shell environment
[NOTE] Pinning command successful, but your installed packages may be out of sync.

I suspect the problem may run deeper. Perhaps I shouldn't have installed opam in my mingw shell, although I vaguely remember just installing in powershell had its own problems (perhaps because I was following the documentation on https://opam.ocaml.org/doc/Install.html without realizing that on windows, I should follow the steps from some blog post instead). Could that have caused the issue? How to uninstall opam on windows? Then perhaps I could reinstall from scratch

[ejgallego]

Hi folks, I ran into this, and I've been tying to investigate, but indeed, so far debugging this hasn't been trivial.

One particular problem is that ocamlfind 1.9.6 doesn't seem like supported on Windows, and pinning to a local ocamlfind repos fails with line ending git issues. But def the problem seems to be in ocamlfind. Also note the OCAMFIND_DEBUG variable.

The problem is that Coq's configure will call:

ocamlfind.exe ocamlc -version

This makes ocamlfind crash with the above exception. Having Coq's configure script just call ocamlc -version makes Coq work again.

Finding the right magic to get ocamlfind print the debug info was not immediate, but I managed in the end:

PS C:\Users\User\coq> $Env:CAMLRUNPARAM="b"
PS C:\Users\User\coq> $Env:OCAMLFIND_DEBUG="b"
PS C:\Users\User\coq> dune exec --root . -- .\tools\configure\configure.exe -no-ask -prefix C:\Users\User\coq\_build\install\default\ -libdir C:\Users\User\coq\_build\install\default\lib\coq
I can not automatically find the name of your architecture.
Give me a name, please [win32 for Win95, Win98 or WinNT]:
win32
Uncaught exception: Invalid_argument("index out of bounds")
Fatal error: exception Invalid_argument("index out of bounds")
Raised by primitive operation at Fl_split.norm_dir.norm_dir_win in file "fl_split.ml", line 132, characters 52-57
Called from Fl_package_base.in_report_search_path.(fun) in file "fl_package_base.ml", line 457, characters 16-35
Called from Stdlib__List.for_all in file "list.ml", line 164, characters 12-15
Called from Stdlib__List.find_all.find in file "list.ml", line 242, characters 17-20
Called from Fl_package_base.module_conflict_report_1 in file "fl_package_base.ml", line 709, characters 4-60
Called from Frontend.conflict_report in file "frontend.ml", line 295, characters 2-74
Called from Frontend.ocamlc in file "frontend.ml", line 1152, characters 2-74
Called from Frontend.main in file "frontend.ml", line 2624, characters 26-41
Called from Frontend in file "frontend.ml", line 2673, characters 2-8
Re-raised at Frontend in file "frontend.ml", line 2681, characters 24-33
Error while running 'C:\Users\User\AppData\Local\opam\4.14.2\bin/ocamlfind.exe ocamlc -version' (exit code 2)
Configuration script failed!
PS C:\Users\User\coq>

[ejgallego]

So indeed it seems https://github.com/ocaml/ocamlfind/pull/64 or a variation of it.

[ejgallego]

I confirm the patch in ocaml/ocamlfind#64 rebased to 1.9.5 works on windows and fixes the issue.

[Plyb]

@ejgallego pardon my ignorance, but how can I get that patch? Using opam pin add https://github.com/db4/ocamlfind.git#win32-fix gets me the same error @JonasOberhauser is seeing

[Plyb]

@ejgallego pardon my ignorance, but how can I get that patch? Using opam pin add https://github.com/db4/ocamlfind.git#win32-fix gets me the same error @JonasOberhauser is seeing

Ah, I figured it out opam pin add https://github.com/ejgallego/ocamlfind.git#1_9_5_fix_win_norm

[ejgallego]

Yes, exactly! That's just the 1.9.5 tree with the fix applied.

We will try to update the findlib version on opam so it includes this patch.

[ejgallego]

Now updated in https://github.com/ocaml/opam-repository/pull/26510

[qcfu-bu]

Now updated in ocaml/opam-repository#26510

after pinning opam pin add ocamlfind --dev, I can compile coq-core but am still running into errors when compiling coq-stdlib. The same thing happens if I pin coq to 8.19.2 as well.

[ERROR] The compilation of coq-stdlib.8.20.0 failed at "dune build -p coq-stdlib -j 15 --promote-install-files=false
        @install".

#=== ERROR while compiling coq-stdlib.8.20.0 ==================================#
# context     2.2.1 | win32/x86_64 | ocaml.5.2.0 | https://opam.ocaml.org#45f961713f04a0d83218c0eb2706131a40f96d54
# path        ~\AppData\Local\opam\default\.opam-switch\build\coq-stdlib.8.20.0
# command     ~\AppData\Local\opam\default\bin\dune.exe build -p coq-stdlib -j 15 --promote-install-files=false @install
# exit-code   1
# env-file    ~\AppData\Local\opam\log\coq-stdlib-12128-d02406.env
# output-file ~\AppData\Local\opam\log\coq-stdlib-12128-d02406.out
### output ###
# [...]
# 696 |   (targets Ltac.timing Ltac.glob Ltac.vos Ltac.vo)
# 697 |   (deps C:/Users/qcfu/AppData/Local/opam/default/lib/coq-core/plugins/ltac\ltac_plugin.cmxs)
# 698 |   (action (chdir ..\..\. (run coqc -noinit -boot -R theories Coq -I C:\Users\qcfu\AppData\Local\opam\default\lib\coq-core\plugins\number_string_notation -I C:\Users\qcfu\AppData\Local\opam\default\lib\coq-core\plugins\zify -I C:\Users\qcfu\AppData\Local\opam\default\lib\coq-core\plugins\tauto -I C:\Users\qcfu\AppData\Local\opam\default\lib\coq-core\plugins\ssreflect -I C:\Users\qcfu\App[...]
# (cd _build/default && C:\Users\qcfu\AppData\Local\opam\default\bin\coqc.exe -noinit -boot -R theories Coq -I C:\Users\qcfu\AppData\Local\opam\default\lib\coq-core\plugins\number_string_notation -I C:\Users\qcfu\AppData\Local\opam\default\lib\coq-core\plugins\zify -I C:\Users\qcfu\AppData\Local\opam\default\lib\coq-core\plugins\tauto -I C:\Users\qcfu\AppData\Local\opam\default\lib\coq-core\plu[...]
# Command exited with code -1073741515.
# File "theories/dune", lines 705-708, characters 1-1697:
# 705 |  (rule
# 706 |   (targets Notations.timing Notations.glob Notations.vos Notations.vo)
# 707 |   (deps )
# 708 |   (action (chdir ..\..\. (run coqc -noinit -boot -R theories Coq -I C:\Users\qcfu\AppData\Local\opam\default\lib\coq-core\plugins\number_string_notation -I C:\Users\qcfu\AppData\Local\opam\default\lib\coq-core\plugins\zify -I C:\Users\qcfu\AppData\Local\opam\default\lib\coq-core\plugins\tauto -I C:\Users\qcfu\AppData\Local\opam\default\lib\coq-core\plugins\ssreflect -I C:\Users\qcfu\App[...]
# (cd _build/default && C:\Users\qcfu\AppData\Local\opam\default\bin\coqc.exe -noinit -boot -R theories Coq -I C:\Users\qcfu\AppData\Local\opam\default\lib\coq-core\plugins\number_string_notation -I C:\Users\qcfu\AppData\Local\opam\default\lib\coq-core\plugins\zify -I C:\Users\qcfu\AppData\Local\opam\default\lib\coq-core\plugins\tauto -I C:\Users\qcfu\AppData\Local\opam\default\lib\coq-core\plu[...]
# Command exited with code -1073741515.

<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><>  🐫
┌─ The following actions failed
│ λ build coq-stdlib 8.20.0
└─
╶─ No changes have been performed

[ejgallego]

The error code seems scary:

This file system error 1073741515 can happen because of missing essential components, wrongly configured system files, faulty hardware, or registry errors.

No idea what is causing this, things do work fine for me. Only idea I have is that maybe you are hitting some kind of path length or command line arguments length limit?