diskuv / dkml-installer-ocaml

The Windows-friendly distribution of OCaml
Apache License 2.0
63 stars 2 forks source link

Missing `advapi32.lib` and `uuid.lib` when compiling with dune #6

Closed panglesd closed 1 year ago

panglesd commented 2 years ago

I did a fresh instal of the dkml distribution of OCaml in the windows virtual machine "MSEdge on win 10" available here.

The install went well, the toplevel works, and dune is installed. However, when I generate a helloworld example using dune init proj helloworld and try to build it, I have some errors about missing .lib files: advapi32.lib and uuid.lib.

Here is the complete output:

PS C:\Users\IEUser\barouf> dune init proj he
Success: initialized project component named he
PS C:\Users\IEUser\barouf> cd he
PS C:\Users\IEUser\barouf\he> dune build
Info: Creating file dune-project with this contents:
| (lang dune 2.9)
| (name he)
    ocamlopt bin/main.exe (exit 2)
(cd _build/default && C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\usr\bin\ocamlopt.opt.exe -w @1..3@5..28@30..39@43@46..47@49..57@61..62-40 -strict-sequence -strict-formats -short-paths -keep-locs -g -o bin/main.exe lib/he.cmxa bin/.main.eobjs/native/dune__exe__Main.cmx)
** Fatal error: Cannot find file "advapi32.lib"
File "caml_startup", line 1:
Error: Error during linking (exit code 2)
    ocamlopt lib/he.cmxs (exit 2)
(cd _build/default && C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\usr\bin\ocamlopt.opt.exe -w @1..3@5..28@30..39@43@46..47@49..57@61..62-40 -strict-sequence -strict-formats -short-paths -keep-locs -g -shared -linkall -I lib -o lib/he.cmxs lib/he.cmxa)
** Fatal error: Cannot find file "uuid.lib"
File "caml_startup", line 1:
Error: Error during linking (exit code 2)
    ocamlopt test/he.exe (exit 2)
(cd _build/default && C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\usr\bin\ocamlopt.opt.exe -w @1..3@5..28@30..39@43@46..47@49..57@61..62-40 -strict-sequence -strict-formats -short-paths -keep-locs -g -o test/he.exe test/.he.eobjs/native/dune__exe__He.cmx)
** Fatal error: Cannot find file "advapi32.lib"
File "caml_startup", line 1:
Error: Error during linking (exit code 2)
jonahbeckford commented 2 years ago

Err ... I can't recreate your situation. I put some mitigations below, but I'd appreciate if you can first copy-and-paste what you get in the Command Prompt when you run:

C:\Users\IEUser\barouf\he>opam switch
C:\Users\IEUser\barouf\he>where.exe dune
C:\Users\IEUser\barouf\he>where.exe dune-real
C:\Users\IEUser\barouf\he>dune clean   
C:\Users\IEUser\barouf\he>set DKML_BUILD_TRACE=ON
C:\Users\IEUser\barouf\he>set DKML_BUILD_TRACE_LEVEL=2
C:\Users\IEUser\barouf\he>dune build

Mitigation 1

That won't work without doing the equivalent of eval $(opam env) on Windows.

I now see I've got some documentation wrong ...

if you are following https://diskuv.gitlab.io/diskuv-ocaml/doc/BeyondBasics.html then:

Open the Command Prompt (press the Windows key ⊞ and R, and then type “cmd” and ENTER).

should really be

Open the x64 Native Tools Command Prompt (press the Windows key ⊞, and then type “x64 native” and ENTER).

Mitigation 2

Or do a opam switch and follow the instructions (ex. for /f "tokens=*" %i in ('opam env') do @%i) in a regular Command Prompt or a PowerShell session.

panglesd commented 2 years ago

Thanks a lot for your answer!

Following your suggestion, I opened the x64 native tool command prompt, dune init proj a new project and built: It worked! Confused, I made more extensive tests, that I now report:

Eval environment, init proj, dune build ```powershell Windows PowerShell Copyright (C) Microsoft Corporation. All rights reserved. PS C:\Users\IEUser> opam switch # switch compiler description C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\dkml ocaml-system.4.12.1 C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\dkml C:\Users\IEUser\switchtest\test1 ocaml-system.4.12.1 C:\Users\IEUser\switchtest\test1 → playground ocaml-system.4.12.1 playground [WARNING] The environment is not in sync with the current switch. You should run: (& opam env) -split '\r?\n' | ForEach-Object { Invoke-Expression $_ } PS C:\Users\IEUser> (& opam env) -split '\r?\n' | ForEach-Object { Invoke-Expression $_ } PS C:\Users\IEUser> opam switch # switch compiler description C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\dkml ocaml-system.4.12.1 C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\dkml C:\Users\IEUser\switchtest\test1 ocaml-system.4.12.1 C:\Users\IEUser\switchtest\test1 → playground ocaml-system.4.12.1 playground PS C:\Users\IEUser> dune init proj proj_powershell Success: initialized project component named proj_powershell PS C:\Users\IEUser> cd proj_powershell PS C:\Users\IEUser\proj_powershell> dune build Info: Creating file dune-project with this contents: | (lang dune 2.9) | (name proj_powershell) ocamlopt bin/main.exe (exit 2) (cd _build/default && C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\usr\bin\ocamlopt.opt.exe -w @1..3@5..28@30..39@43@46..47@49..57@61..62-40 -strict-sequence -strict-formats -short-paths -keep-locs -g -o bin/main.exe lib/proj_powershell.cmxa bin/.main.eobjs/native/dune__exe__Main.cmx) ** Fatal error: Cannot find file "advapi32.lib" File "caml_startup", line 1: Error: Error during linking (exit code 2) ocamlopt lib/proj_powershell.cmxs (exit 2) (cd _build/default && C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\usr\bin\ocamlopt.opt.exe -w @1..3@5..28@30..39@43@46..47@49..57@61..62-40 -strict-sequence -strict-formats -short-paths -keep-locs -g -shared -linkall -I lib -o lib/proj_powershell.cmxs lib/proj_powershell.cmxa) ** Fatal error: Cannot find file "uuid.lib" File "caml_startup", line 1: Error: Error during linking (exit code 2) ocamlopt test/proj_powershell.exe (exit 2) (cd _build/default && C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\usr\bin\ocamlopt.opt.exe -w @1..3@5..28@30..39@43@46..47@49..57@61..62-40 -strict-sequence -strict-formats -short-paths -keep-locs -g -o test/proj_powershell.exe test/.proj_powershell.eobjs/native/dune__exe__Proj_powershell.cmx) ** Fatal error: Cannot find file "advapi32.lib" File "caml_startup", line 1: Error: Error during linking (exit code 2) PS C:\Users\IEUser\proj_powershell> ``` (Similar output for a cmd, except with `for /f "tokens=*" %i in ('opam env') do @%i` in place of `(& opam env) -split '\r?\n' | ForEach-Object { Invoke-Expression $_ }`).
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

PS C:\Users\IEUser> mkdir newswitchtest

    Directory: C:\Users\IEUser

Mode                LastWriteTime         Length Name
----                -------------         ------ ----
d-----        8/29/2022   1:08 AM                newswitchtest

PS C:\Users\IEUser> cd newswitchtest
PS C:\Users\IEUser\newswitchtest> opam dkml init
[NOTE] External dependency handling not supported for OS family 'windows'.
       You can disable this check using 'opam option --global depext=false'
Opam plugin "dkml" may require upgrading/reinstalling. Reinstall the plugin on the current switch? [y/n] y
[NOTE] External dependency handling not supported for OS family 'windows'.
       You can disable this check using 'opam option --global depext=false'
The following actions will be performed:
=== recompile 1 package
  ↕ opam-dkml 1.0.0 (pinned)

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
λ removed   opam-dkml.1.0.0
✶ installed opam-dkml.1.0.0
Done.
# Run eval $(opam env) to update the current shell environment

<><> Carrying on to "C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\bin\opam-real.exe dkml init"

The compiler switch C:\Users\IEUser\newswitchtest does not exist.

<><> Installing new switch packages <><><><><><><><><><><><><><><><><><><><><><>
Switch invariant: ["ocaml-system" {= "4.12.1"} "conf-withdkml"]
[NOTE] External dependency handling not supported for OS family 'windows'.
       You can disable this check using 'opam option --global depext=false'

<><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><>
▼ retrieved conf-withdkml.1  (cached)
✶ installed base-bigarray.base
✶ installed base-threads.base
✶ installed base-unix.base
✶ installed conf-withdkml.1
✶ installed ocaml-system.4.12.1
✶ installed ocaml-config.3
✶ installed ocaml.4.12.1
Done.
# Run eval $(opam env '--switch=C:\Users\IEUser\newswitchtest') to update the current shell environment
Added 'PATH += "C:\\Users\\IEUser\\AppData\\Local\\Programs\\DISKUV~1\\0\\usr\\bin"' to field setenv in switch C:\Users\IEUser\newswitchtest
Set to '["C:\\Users\\IEUser\\AppData\\Local\\Programs\\DISKUV~1\\0\\dkml\\_opam\\bin\\with-dkml.exe"]' the field wrap-build-commands in switch C:\Users\IEUser\newswitchtest
Set to '["C:\\Users\\IEUser\\AppData\\Local\\Programs\\DISKUV~1\\0\\dkml\\_opam\\bin\\with-dkml.exe"]' the field wrap-install-commands in switch C:\Users\IEUser\newswitchtest
Set to '["C:\\Users\\IEUser\\AppData\\Local\\Programs\\DISKUV~1\\0\\dkml\\_opam\\bin\\with-dkml.exe"]' the field wrap-remove-commands in switch C:\Users\IEUser\newswitchtest
Set to '[]' the field pre-build-commands in switch C:\Users\IEUser\newswitchtest
Set to '[]' the field post-install-commands in switch C:\Users\IEUser\newswitchtest
Set to '[]' the field pre-remove-commands in switch C:\Users\IEUser\newswitchtest
PS C:\Users\IEUser\newswitchtest> dune init proj projtest
Success: initialized project component named projtest
PS C:\Users\IEUser\newswitchtest> cd projtest
PS C:\Users\IEUser\newswitchtest\projtest> dune build
Info: Creating file dune-project with this contents:
| (lang dune 2.9)
| (name projtest)
PS C:\Users\IEUser\newswitchtest\projtest>

Eval (playground) environment, create new switch, init project, build fails, eval new switch env, build succeed ```powershell Windows PowerShell Copyright (C) Microsoft Corporation. All rights reserved. PS C:\Users\IEUser> opam switch # switch compiler description C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\dkml ocaml-system.4.12.1 C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\dkml C:\Users\IEUser\newswitchtest ocaml-system.4.12.1 C:\Users\IEUser\newswitchtest C:\Users\IEUser\switchtest\test1 ocaml-system.4.12.1 C:\Users\IEUser\switchtest\test1 → playground ocaml-system.4.12.1 playground [WARNING] The environment is not in sync with the current switch. You should run: (& opam env) -split '\r?\n' | ForEach-Object { Invoke-Expression $_ } PS C:\Users\IEUser> (& opam env) -split '\r?\n' | ForEach-Object { Invoke-Expression $_ } PS C:\Users\IEUser> mkdir switch_experiment Directory: C:\Users\IEUser Mode LastWriteTime Length Name ---- ------------- ------ ---- d----- 8/29/2022 1:15 AM switch_experiment PS C:\Users\IEUser> cd switch_experiment PS C:\Users\IEUser\switch_experiment> opam dkml init [NOTE] External dependency handling not supported for OS family 'windows'. You can disable this check using 'opam option --global depext=false' Opam plugin "dkml" may require upgrading/reinstalling. Reinstall the plugin on the current switch? [y/n] y [NOTE] External dependency handling not supported for OS family 'windows'. You can disable this check using 'opam option --global depext=false' The following actions will be performed: === recompile 1 package ↕ opam-dkml 1.0.0 (pinned) <><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><> λ removed opam-dkml.1.0.0 ✶ installed opam-dkml.1.0.0 Done. <><> Carrying on to "C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\bin\opam-real.exe dkml init" The compiler switch C:\Users\IEUser\switch_experiment does not exist. <><> Installing new switch packages <><><><><><><><><><><><><><><><><><><><><><> Switch invariant: ["ocaml-system" {= "4.12.1"} "conf-withdkml"] [NOTE] External dependency handling not supported for OS family 'windows'. You can disable this check using 'opam option --global depext=false' <><> Processing actions <><><><><><><><><><><><><><><><><><><><><><><><><><><><> ▼ retrieved conf-withdkml.1 (cached) ✶ installed base-bigarray.base ✶ installed base-threads.base ✶ installed base-unix.base ✶ installed conf-withdkml.1 ✶ installed ocaml-system.4.12.1 ✶ installed ocaml-config.3 ✶ installed ocaml.4.12.1 Done. # Run eval $(opam env '--switch=C:\Users\IEUser\switch_experiment') to update the current shell environment Added 'PATH += "C:\\Users\\IEUser\\AppData\\Local\\Programs\\DISKUV~1\\0\\usr\\bin"' to field setenv in switch C:\Users\IEUser\switch_experiment Set to '["C:\\Users\\IEUser\\AppData\\Local\\Programs\\DISKUV~1\\0\\dkml\\_opam\\bin\\with-dkml.exe"]' the field wrap-build-commands in switch C:\Users\IEUser\switch_experiment Set to '["C:\\Users\\IEUser\\AppData\\Local\\Programs\\DISKUV~1\\0\\dkml\\_opam\\bin\\with-dkml.exe"]' the field wrap-install-commands in switch C:\Users\IEUser\switch_experiment Set to '["C:\\Users\\IEUser\\AppData\\Local\\Programs\\DISKUV~1\\0\\dkml\\_opam\\bin\\with-dkml.exe"]' the field wrap-remove-commands in switch C:\Users\IEUser\switch_experiment Set to '[]' the field pre-build-commands in switch C:\Users\IEUser\switch_experiment Set to '[]' the field post-install-commands in switch C:\Users\IEUser\switch_experiment Set to '[]' the field pre-remove-commands in switch C:\Users\IEUser\switch_experiment PS C:\Users\IEUser\switch_experiment> dune build PS C:\Users\IEUser\switch_experiment> dune init proj exp Success: initialized project component named exp PS C:\Users\IEUser\switch_experiment> cd exp PS C:\Users\IEUser\switch_experiment\exp> dune build Info: Creating file dune-project with this contents: | (lang dune 2.9) | (name exp) ocamlopt bin/main.exe (exit 2) (cd _build/default && C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\usr\bin\ocamlopt.opt.exe -w @1..3@5..28@30..39@43@46..47@49..57@61..62-40 -strict-sequence -strict-formats -short-paths -keep-locs -g -o bin/main.exe lib/exp.cmxa bin/.main.eobjs/native/dune__exe__Main.cmx) ** Fatal error: Cannot find file "advapi32.lib" File "caml_startup", line 1: Error: Error during linking (exit code 2) ocamlopt lib/exp.cmxs (exit 2) (cd _build/default && C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\usr\bin\ocamlopt.opt.exe -w @1..3@5..28@30..39@43@46..47@49..57@61..62-40 -strict-sequence -strict-formats -short-paths -keep-locs -g -shared -linkall -I lib -o lib/exp.cmxs lib/exp.cmxa) ** Fatal error: Cannot find file "uuid.lib" File "caml_startup", line 1: Error: Error during linking (exit code 2) ocamlopt test/exp.exe (exit 2) (cd _build/default && C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\usr\bin\ocamlopt.opt.exe -w @1..3@5..28@30..39@43@46..47@49..57@61..62-40 -strict-sequence -strict-formats -short-paths -keep-locs -g -o test/exp.exe test/.exp.eobjs/native/dune__exe__Exp.cmx) ** Fatal error: Cannot find file "advapi32.lib" File "caml_startup", line 1: Error: Error during linking (exit code 2) PS C:\Users\IEUser\switch_experiment\exp> opam switch # switch compiler description C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\dkml ocaml-system.4.12.1 C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\dkml C:\Users\IEUser\newswitchtest ocaml-system.4.12.1 C:\Users\IEUser\newswitchtest → C:\Users\IEUser\switch_experiment ocaml-system.4.12.1 C:\Users\IEUser\switch_experiment C:\Users\IEUser\switchtest\test1 ocaml-system.4.12.1 C:\Users\IEUser\switchtest\test1 playground ocaml-system.4.12.1 playground [NOTE] Current switch has been selected based on the current directory. The current global system switch is playground. [WARNING] The environment is not in sync with the current switch. You should run: (& opam env) -split '\r?\n' | ForEach-Object { Invoke-Expression $_ } PS C:\Users\IEUser\switch_experiment\exp> (& opam env) -split '\r?\n' | ForEach-Object { Invoke-Expression $_ } PS C:\Users\IEUser\switch_experiment\exp> dune build PS C:\Users\IEUser\switch_experiment\exp> ```

So it seems that evaluating the environment to the playground switch (after a successfull build in a newly created switch, I tried to go back to the playground switch with (& opam env --switch=playground) -split '\r?\n' | ForEach-Object { Invoke-Expression $_ }) clean and build again, and it failed!

(By the way, the documentation says:

Notice how the switch was created with opam dkml init, and also notice how opam switch tells you in its [NOTE] that it knows which switch should be used based on the current directory.

If we want our my-first-switch to be remembered regardless what the directory currently is, we can follow the [WARNING] and add the option --set-switch.

But, from the last experiment, it seems that even if we are in the directory, if we do not update the environment, it won't use the local switch...

Also, when creating a new switch with opam dkml init, the message is to use eval $(opam env), which won't work on windows...

PS C:\Users\IEUser\switch_experiment> opam dkml init
[...]
✶ installed ocaml.4.12.1
Done.
# Run eval $(opam env '--switch=C:\Users\IEUser\switch_experiment') to update the current shell environment
Added 'PATH += "C:\\Users\\IEUser\\AppData\\Local\\Programs\\DISKUV~1\\0\\usr\\bin"' to field setenv in switch C:\Users\IEUser\switch_experiment
[...]

)

panglesd commented 2 years ago

And here is the result of the command you sent me, in the context of a failing build (playground switch):

PS C:\Users\IEUser\barouf\he> opam switch
#  switch                                                  compiler             description
   C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\dkml  ocaml-system.4.12.1  C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\dkml
→  playground                                              ocaml-system.4.12.1  playground
PS C:\Users\IEUser\barouf\he> where.exe dune
C:\Users\IEUser\AppData\Local\opam\playground\bin\dune.exe
C:\Users\IEUser\AppData\Local\Programs\DiskuvOCaml\0\usr\bin\dune.exe
PS C:\Users\IEUser\barouf\he> where.exe dune-real
C:\Users\IEUser\AppData\Local\Programs\DiskuvOCaml\0\usr\bin\dune-real.exe
PS C:\Users\IEUser\barouf\he> dune clean
PS C:\Users\IEUser\barouf\he> set DKML_BUILD_TRACE=on
PS C:\Users\IEUser\barouf\he> set DKML_BUILD_TRACE_LEVEL=2
PS C:\Users\IEUser\barouf\he> dune build
    ocamlopt bin/main.exe (exit 2)
(cd _build/default && C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\usr\bin\ocamlopt.opt.exe -w @1..3@5..28@30..39@43@46..47@49..57@61..62-40 -strict-sequence -strict-formats -short-paths -keep-locs -g -o bin/main.exe lib/he.cmxa bin/.main.eobjs/native/dune__exe__Main.cmx)
** Fatal error: Cannot find file "advapi32.lib"
File "caml_startup", line 1:
Error: Error during linking (exit code 2)
    ocamlopt lib/he.cmxs (exit 2)
(cd _build/default && C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\usr\bin\ocamlopt.opt.exe -w @1..3@5..28@30..39@43@46..47@49..57@61..62-40 -strict-sequence -strict-formats -short-paths -keep-locs -g -shared -linkall -I lib -o lib/he.cmxs lib/he.cmxa)
** Fatal error: Cannot find file "uuid.lib"
File "caml_startup", line 1:
Error: Error during linking (exit code 2)
    ocamlopt test/he.exe (exit 2)
(cd _build/default && C:\Users\IEUser\AppData\Local\Programs\DISKUV~1\0\usr\bin\ocamlopt.opt.exe -w @1..3@5..28@30..39@43@46..47@49..57@61..62-40 -strict-sequence -strict-formats -short-paths -keep-locs -g -o test/he.exe test/.he.eobjs/native/dune__exe__He.cmx)
** Fatal error: Cannot find file "advapi32.lib"
File "caml_startup", line 1:
Error: Error during linking (exit code 2)
jonahbeckford commented 2 years ago

Thanks for the detailed report!! I missed that you were using PowerShell, so the trace statements did not work.

Is it possible that you can repeat the following commands in PowerShell ... you will get a lot of output:

dune clean

Remove-Item -Recurse "$(opam var prefix)\.dkml\compiler-cache"

$env:DKML_BUILD_TRACE="ON"
$env:DKML_BUILD_TRACE_LEVEL=2
dune build
jonahbeckford commented 2 years ago

Bugs to be fixed (so far):

panglesd commented 2 years ago

Is it possible that you can repeat the following commands in PowerShell ... you will get a lot of output:

It is a bit strange... Actually I do not get any output, apart from the copy-pasted error...

Which made me remark that the usual dune information are also not displayed (the kind of "progress bar). It is as if the standard output were redirected elsewhere...

This happens only in the faulty environment (powershell or bash with environment synced with playground).

I made a recording of what happens (you can ignore the first 40 seconds where my machine was lagging a bit) MSEdge - Win10.webm

jonahbeckford commented 2 years ago

Thanks for the recording! I can recreate the problem now. This will take a bit of time to figure out the solution.

jonahbeckford commented 2 years ago

There was a race condition during installation of the Dune shim; the compilation was changing based on the existence of an external file. I'm testing and packaging up Diskuv OCaml 1.0.1 which will fix this issue permanently.

For now, running the following in PowerShell inside any broken switch (ex. playground) should fix it:

opam repository set-url diskuv-1.0.0 git+https://github.com/diskuv/diskuv-opam-repository.git#v1.0.1-prerel1
opam pin add dune -k version 2.9.3+shim.1.0.1
jonahbeckford commented 2 years ago

@panglesd The new Diskuv OCaml 1.0.1 fixes this bug. I've updated the ocaml.org PR as well. https://github.com/diskuv/dkml-installer-ocaml/releases/download/v1.0.1/setup-diskuv-ocaml-windows_x86_64-1.0.1.exe

panglesd commented 2 years ago

Thanks a lot! I wanted to test your workaround but for some reason the virtual machine refused to launch... I'll definitely check the new diskuv version!

jonahbeckford commented 1 year ago

Doesn't look like a problem anymore. Can re-open if it still is an issue.