search

ocaml / flexdll

a dlopen-like API for Windows
Other
97 stars 30 forks source link

`-print-search-dirs` may return `;`-delimited mixed paths on MSYS2 #97

Closed dra27 closed 1 year ago

dra27 commented 2 years ago

Partially a TODO item to check, unless someone else can confirm/disprove in the meantime, as I don't have a relevantly-configured machine to hand.

in: https://github.com/alainfrisch/flexdll/blob/3255266d71813572b771f4946ed932912c760a7c/reloc.ml#L1260-L1263 there are several assumptions:

Both of these assumptions are valid for Cygwin-compiled mingw compilers, but I'm not sure they are valid for MSYS2, where gcc is (can be?) a native Windows executable and returns ;-separated mixed-mode paths.

dra27 commented 2 years ago

cc @jonahbeckford just in case you've already seen this?

yakobowski commented 2 years ago

FWIW, I have been meaning to ask a very related question for years. Namely would it be possible to support the "native" MinGW compiler, and not just the one packaged with Cygwin? We have been carrying a patch that changes : into ; in get_lib_search_dirs for years, and the build works fine with just this change. I never quite understood where/how I could have detected whether we are using a "full" MinGW compiler, or the Cygwin one.

(I cannot directly comment on MSYS2, so apologies for this partial hijack :sweat_smile: . The context of this question is that we are using a MinGW GCC which we build ourselves, which uses ;-separated paths. But the directory delimiter are /, so we did not have to patch normalize_paths... :dizzy_face: )

jonahbeckford commented 2 years ago

(There are responses to both queries below)

@dra27

(I don't see this issue because I don't generally use gcc on Windows)

On MSYS2 the normal gcc (which is installed with pacman -S msys/gcc) behaves like Cygwin:

beckf@DESKTOP MSYS /z/Temp
$ /usr/bin/gcc -print-search-dirs
install: /usr/lib/gcc/x86_64-pc-msys/11.2.0/
programs: =/usr/lib/gcc/x86_64-pc-msys/11.2.0/:/usr/lib/gcc/x86_64-pc-msys/11.2.0/:/usr/lib/gcc/x86_64-pc-msys/:/usr/lib/gcc/x86_64-pc-msys/11.2.0/:/usr/lib/gcc/x86_64-pc-msys/:/usr/lib/gcc/x86_64-pc-msys/11.2.0/../../
../../x86_64-pc-msys/bin/x86_64-pc-msys/11.2.0/:/usr/lib/gcc/x86_64-pc-msys/11.2.0/../../../../x86_64-pc-msys/bin/
libraries: =/usr/lib/gcc/x86_64-pc-msys/11.2.0/:/usr/lib/gcc/x86_64-pc-msys/11.2.0/../../../../x86_64-pc-msys/lib/x86_64-pc-msys/11.2.0/:/usr/lib/gcc/x86_64-pc-msys/11.2.0/../../../../x86_64-pc-msys/lib/../lib/:/usr/li
b/gcc/x86_64-pc-msys/11.2.0/../../../x86_64-pc-msys/11.2.0/:/usr/lib/gcc/x86_64-pc-msys/11.2.0/../../../../lib/:/lib/x86_64-pc-msys/11.2.0/:/lib/../lib/:/usr/lib/x86_64-pc-msys/11.2.0/:/usr/lib/../lib/:/usr/lib/gcc/x86
_64-pc-msys/11.2.0/../../../../x86_64-pc-msys/lib/:/usr/lib/gcc/x86_64-pc-msys/11.2.0/../../../:/lib/:/usr/lib/

beckf@DESKTOP MSYS /z/Temp
$ ldd /usr/bin/gcc
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffada9a0000)
        KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7ffad9840000)
        KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7ffad7f80000)
        msys-iconv-2.dll => /usr/bin/msys-iconv-2.dll (0x5603f0000)
        msys-intl-8.dll => /usr/bin/msys-intl-8.dll (0x430b30000)
        msys-2.0.dll => /usr/bin/msys-2.0.dll (0x180040000)

@dra27 @yakobowski

The MinGW gcc (which is installed with pacman -S mingw64/mingw-w64-x86_64-gcc or pacman -S mingw64/mingw-w64-x86_64-riscv64-unknown-elf-gcc, etc.) is a native Win32 executable:

beckf@DESKTOP MSYS /z/Temp
$ /mingw64/bin/x86_64-w64-mingw32-gcc.exe -print-search-dirs
install: C:/Users/beckf/AppData/Local/Programs/DiskuvOCaml/1/tools/MSYS2/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/11.2.0/
programs: =C:/Users/beckf/AppData/Local/Programs/DiskuvOCaml/1/tools/MSYS2/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/11.2.0/;C:/Users/beckf/AppData/Local/Programs/DiskuvOCaml/1/tools/MSYS2/mingw64/bin/../lib/gcc/;C:/Us
ers/beckf/AppData/Local/Programs/DiskuvOCaml/1/tools/MSYS2/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../x86_64-w64-mingw32/bin/x86_64-w64-mingw32/11.2.0/;C:/Users/beckf/AppData/Local/Programs/DiskuvOCam
l/1/tools/MSYS2/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../x86_64-w64-mingw32/bin/
libraries: =C:/Users/beckf/AppData/Local/Programs/DiskuvOCaml/1/tools/MSYS2/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/11.2.0/;C:/Users/beckf/AppData/Local/Programs/DiskuvOCaml/1/tools/MSYS2/mingw64/bin/../lib/gcc/;C:/U
sers/beckf/AppData/Local/Programs/DiskuvOCaml/1/tools/MSYS2/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../x86_64-w64-mingw32/lib/x86_64-w64-mingw32/11.2.0/;C:/Users/beckf/AppData/Local/Programs/DiskuvOCa
ml/1/tools/MSYS2/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../x86_64-w64-mingw32/lib/../lib/;C:/Users/beckf/AppData/Local/Programs/DiskuvOCaml/1/tools/MSYS2/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/11.
2.0/../../../x86_64-w64-mingw32/11.2.0/;C:/Users/beckf/AppData/Local/Programs/DiskuvOCaml/1/tools/MSYS2/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../lib/;D:/a/msys64/mingw64/x86_64-w64-mingw32/lib/x86_6
4-w64-mingw32/11.2.0/;D:/a/msys64/mingw64/x86_64-w64-mingw32/lib/../lib/;C:/Users/beckf/AppData/Local/Programs/DiskuvOCaml/1/tools/MSYS2/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../../x86_64-w64-mingw32/l
ib/;C:/Users/beckf/AppData/Local/Programs/DiskuvOCaml/1/tools/MSYS2/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/11.2.0/../../../;D:/a/msys64/mingw64/x86_64-w64-mingw32/lib/

beckf@DESKTOP MSYS /z/Temp
$ ldd /mingw64/bin/x86_64-w64-mingw32-gcc.exe
        ntdll.dll => /c/WINDOWS/SYSTEM32/ntdll.dll (0x7ffada9a0000)
        KERNEL32.DLL => /c/WINDOWS/System32/KERNEL32.DLL (0x7ffad9840000)
        KERNELBASE.dll => /c/WINDOWS/System32/KERNELBASE.dll (0x7ffad7f80000)
        msvcrt.dll => /c/WINDOWS/System32/msvcrt.dll (0x7ffada6f0000)
        libwinpthread-1.dll => /mingw64/bin/libwinpthread-1.dll (0x7ffac23e0000)

@yakobowski

  1. If you are trying to see whether an executable is a native Windows executable (for example, you are trying to answer whether you should give the executable native Windows paths), you can run ldd just like I did above and check for the presence of msys-2.0.dll or cygwin1.dll. There is a native Windows equivalent of ldd in a location like C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Tools\MSVC\14.26.28801\bin\HostX64\x64\dumpbin.exe /dependents
  2. If you are trying to see whether your environment inherited by your parent process is Cygwin or MSYS2 (for example, you are in a Makefile and are trying to find out if you are in a Cygwin or MSYS2 make.exe or in native Windows nmake.exe), generally all you have to do is check for the presence of cygpath in your PATH. If it is present you are in Cygwin or MSYS2 environment.
dra27 commented 2 years ago

Thankyou both! (and definitely not a hijack, @yakobowski!).

OpamStd.Sys.is_cygwin_variant does a similar analysis using cygcheck, but this could be slightly simpler in this case (which is good, because flexlink can't depend on the Unix library, which is why it has some funny dances with temporary files). I think it would be sufficient in this case just to classify the install: path while parsing the output of -print-search-dirs.

I expect it's reasonable to assume that the paths will only ever use forward slashes!

Patches definitely welcome - even more so if they come with a CI check 🙂

yakobowski commented 2 years ago

The awful truth is that our compiles emits forward slashes everywhere in the output of -print-search-dirs except for install: which is a terrible mix of both. But this may be because this is taken straight from the compilation line, and I don't think we care about this line anyway.

install: c:\home\boris\wave\x86_64-windows64\stable-gnat\install\bin\../lib/gcc/x86_64-w64-mingw32/11.2.1/
programs: =c:/home/boris/wave/x86_64-windows64/stable-gnat/install/bin/../libexec/gcc/x86_64-w64-mingw32/11.2.1/;c:/home/boris/wave/x86_64-windows64/stable-gnat/install/bin/../libexec/gcc/;c:/home/boris/wave/x86_64-windows64/stable-gnat/install/bin/../lib/gcc/x86_64-w64-mingw32/11.2.1/../../../../x86_64-w64-mingw32/bin/x86_64-w64-mingw32/11.2.1/;c:/home/boris/wave/x86_64-windows64/stable-gnat/install/bin/../lib/gcc/x86_64-w64-mingw32/11.2.1/../../../../x86_64-w64-mingw32/bin/
libraries: =c:/home/boris/wave/x86_64-windows64/stable-gnat/install/bin/../lib/gcc/x86_64-w64-mingw32/11.2.1/;c:/home/boris/wave/x86_64-windows64/stable-gnat/install/bin/../lib/gcc/;c:/home/boris/wave/x86_64-windows64/stable-gnat/install/bin/../lib/gcc/x86_64-w64-mingw32/11.2.1/../../../../x86_64-w64-mingw32/lib/x86_64-w64-mingw32/11.2.1/;c:/home/boris/wave/x86_64-windows64/stable-gnat/install/bin/../lib/gcc/x86_64-w64-mingw32/11.2.1/../../../../x86_64-w64-mingw32/lib/../lib/;c:/home/boris/wave/x86_64-windows64/stable-gnat/install/bin/../lib/gcc/x86_64-w64-mingw32/11.2.1/../../../x86_64-w64-mingw32/11.2.1/;c:/home/boris/wave/x86_64-windows64/stable-gnat/install/bin/../lib/gcc/x86_64-w64-mingw32/11.2.1/../../../../lib/;/mingw/lib/x86_64-w64-mingw32/11.2.1/;/mingw/lib/../lib/;c:/home/boris/wave/x86_64-windows64/stable-gnat/install/bin/../lib/gcc/x86_64-w64-mingw32/11.2.1/../../../../x86_64-w64-mingw32/lib/;c:/home/boris/wave/x86_64-windows64/stable-gnat/install/bin/../lib/gcc/x86_64-w64-mingw32/11.2.1/../../../;/mingw/lib/
dra27 commented 2 years ago

I meant that the search paths (programs and libraries) would only use forward slashes! The install one is fine - the main thing is it can be classified as a Windows path.

dra27 commented 2 years ago

(i.e. the c: is the important bit)

yakobowski commented 1 year ago

I did not manage to test #118 until until today (after it was merged), but I can confirm that I successfully built OCaml 4.14 with the latest flexdll and no other patch whatsoever with just ./configure --host=x86_64-w64-mingw32. This was with "our" x86_64-w64-mingw32-gcc, which uses ; as separator, and was built in a Cygwin env (to add more confusion). So THANK YOU for all the patches.

dra27 commented 1 year ago

My pleasure, @yakobowski - sorry it took quite so long to get to! I will be doing a release of flexdll with it fairly soon (either this week or perhaps next).