Compiling under Windows: unescaped unicode characters in paths

[kotsius]

https://elixirforum.com/t/argon2-elixir-on-windows-10-running-out-of-ideas/26156

I hit a brick wall when argon2_elixir compilation started failing, due to a change in the Erlang path that included a non-ASCII character. This should be the simplest of fixes ("They need to use ~ts instead of ~s when printing strings in a unicode environment."), but can spare future users from an awful lot of pain. Thanks for this indispensible dependency!

[riverrun]

For future reference, this rebar issue also references this problem.

[riverrun]

Thanks for raising the issue.

I have pushed some changes to the unicode_path_compilation branch. Could you test that and make sure it works properly?

[kotsius]

Now I am getting:

Copyright (C) Microsoft Corporation. All rights reserved. del /Q /F priv Could Not Find d:\Websites\Projects\MintFirsts\deps\argon2_elixir\priv erl -eval "io:format(\"~ts~n\", [lists:concat([\"ERTS_INCLUDE_PATH=\", code:root_dir(), \"/erts-\", erlang:system_info(version), \"/include\"])])" -s init stop -noshell > Makefile.auto.win nmake / /F Makefile.win priv\argon2_nif.dll Microsoft (R) Program Maintenance Utility Version 14.16.27034.0 Copyright (C) Microsoft Corporation. All rights reserved. if NOT EXIST "priv" mkdir "priv" cl /O2 /EHsc /I"argon2\include" /I"argon2\src" /I"c:/Program Files/\x{2022} Programming/Erlang/erts-10.5/include" /LD /MD /Fepriv\argon2_nif.dll argon2\src\argon2.c argon2\src\core.c argon2\src\blake2\blake2b.c argon2\src\thread.c argon2\src\encoding.c argon2\src\ref.c c_src\argon2_nif.c Microsoft (R) C/C++ Optimizing Compiler Version 19.16.27034 for x64 Copyright (C) Microsoft Corporation. All rights reserved. argon2.c c1: fatal error C1083: Cannot open source file: 'argon2\src\argon2.c': No such file or directory core.c c1: fatal error C1083: Cannot open source file: 'argon2\src\core.c': No such file or directory blake2b.c [...]

This could well be due to my non-experience in using anything other than the hex.pm packages directly. What I did (other than reinstating the unicode character in the erl path) was to change the mix.exs entry to:

{:argon2_elixir, "~> 2.0", branch: "unicode_path_compilation", github: "riverrun/argon2_elixir"}

[riverrun]

I think the problem is with how it deals with the submodule. I will look into how to sort this out and get back to you later today.

[riverrun]

Ok. Try adding the following to the deps section of your mix.exs file:

{:argon2_elixir, "~> 2.0", github: "riverrun/argon2_elixir", branch: "unicode_path_compilation", submodules: true}

If that doesn't work, edit the Makefile.win in the deps/argon2_elixir directory, changing the ~s to ~ts, and see if that compiles.

[kotsius]

1st try: submodules option ➔ cannot download

2nd try: without submodules option ➔ does not compile even though ~ts already in place

3nd try: regular argon2_elixir (master) + change ~s to ~ts ➔ again, does not compile

[riverrun]

I have just pushed the changed to hex. Please try version 2.1.1 and let me know if it works.

[kotsius]

Cleared _build and deps, then got this:

[riverrun]

Could you do the following two things and then report back?

Find out where the erl_nif.h file is.

In the cmd shell, run erl -eval "io:format(\"~ts~n\", [lists:concat([\"ERTS_INCLUDE_PATH=\", code:root_dir(), \"/erts-\", erlang:system_info(version), \"/include\"])])" -s init stop -noshell and let me know the output.

[kotsius]

erl_nif.h found in 2 locations:

[program files path]\Erlang\erts-10.5\include [program files path]\Erlang\usr\include

The command outputs:

ERTS_INCLUDE_PATH=c:/Program Files/\x{2022} Programming/Erlang/erts-10.5/include

[riverrun]

And if you type DIR c:/Program Files/\x{2022} Programming/Erlang/erts-10.5/include, what is the output?

[kotsius]

Parameter format not correct - "Program".

[riverrun]

Try entering the erlang shell, by typing erl, and then in the erlang shell, run io:format(\"~ts~n\", [lists:concat([\"ERTS_INCLUDE_PATH=\", code:root_dir(), \"/erts-\", erlang:system_info(version), \"/include\"])])

[kotsius]

No output, seems hung.

[riverrun]

Sorry, I forgot to mention the . at the end of the command. Just type the above command with . at the end, and it should work.

[kotsius]

No problem, yet still hanging.

[riverrun]

Let's try something else.

Try running erl -noshell -s init stop -eval "io:format(\"~ts/erts-~ts/include/\", [code:root_dir(), erlang:system_info(version)])." in the cmd shell.

[kotsius]

Output = c:/Program Files/\x{2022} Programming/Erlang/erts-10.5/include/

[riverrun]

Ok, let's try erl -noshell -s init stop -eval "io:setopts(standard_io, [{encoding, unicode}]), io:format(\"ERTS_INCLUDE_PATH=~ts/erts-~ts/include/\", [code:root_dir(), erlang:system_info(version)])." in the cmd shell.

[kotsius]

output = ERTS_INCLUDE_PATH=c:/Program Files/• Programming/Erlang/erts-10.5/include/

[riverrun]

That's better. I will release a new version soon. I'll let you know when it has been pushed to hex.

[kotsius]

Great.

[riverrun]

Just released 2.1.2. Let's hope that works :)

[kotsius]

One has to admire this bug's perseverance!

[riverrun]

And erl_nif.h is definitely in the c:/Program Files/• Programming/Erlang/erts-10.5/include/ directory?

[kotsius]

Yes, it certainly seems so.

[riverrun]

I will look into it again next week. If you can find out any further information that could help us fix this, that would be great.

[kotsius]

If this issue is too much trouble to resolve, perhaps a simple warning that the Erlang path should not contain unicode characters would suffice. Otherwise, I am happy to help with further testing.

[riverrun]

At the moment, are you able to use argon2_elixir if you move the entire Erlang directory to a different path?

[kotsius]

When the unicode character (• plus a space) is replaced with an ascii one (+ minus the space), it compiles. This is the output:

[Whizboy-Arnold]

WOW! how do you guys even manage to compile on windows, haven't even been able to get past 1st step, i have installed cpp tools with choco, vs installer, mingw. still hitting a hard wall.

[kotsius]

WOW! how do you guys even manage to compile on windows, haven't even been able to get past 1st step, i have installed cpp tools with choco, vs installer, mingw. still hitting a hard wall.

My setup is only based on Visual Studio, community version 2022. According to my own notes, it has to include "C++ core desktop features", "MSVC […] VS 2022 C++ x64/x86 build […]" and "Windows 10 SDK [most recent version]" (since I am on Windows 10).

Also, something like C:\path\to\Visual Studio\VC\Tools\MSVC\14.36.32532\bin\Hostx64\x64 must be added to PATH (Erlang is a x64 application).

Hope this helps.

[riverrun]

Closing this issue. If anyone feels like reopening, please let me know.