search

mesonbuild / meson

The Meson Build System
http://mesonbuild.com
Apache License 2.0
5.58k stars 1.62k forks source link

add_install_script() / ninja install fail with "[script.sh]: No such file or directory" / "FAILED: install script 'bash [script.sh]'" #8565

Open db-src opened 3 years ago

db-src commented 3 years ago

Describe the bug ninja install does not work on MSYS2/MinGW64. It 'cannot find' even the simplest install script.

$ ninja install
[0/1] Installing files.
Running custom install script 'bash C:/msys64/home/user.name/src/tmp/test.sh HELLO'
--- stdout ---

--- stderr ---
/bin/bash: C:/msys64/home/user.name/src/tmp/test.sh: No such file or directory

FAILED: install script 'bash C:/msys64/home/user.name/src/tmp/test.sh HELLO' exit code 127, stopped
FAILED: meson-install
"C:/msys64/mingw64/bin/meson" "install" "--no-rebuild"
ninja: build stopped: subcommand failed.

To Reproduce meson.build:

project('test')
meson.add_install_script('test.sh')

tesh.sh:

#!/bin/bash
echo hiya

Expected behavior The script should be trivially found and run!

system parameters

db-src commented 3 years ago

Still a problem with 0.57.2. Can I really be the only person experiencing this? It seems a little, err, totally breaking.

xclaesse commented 3 years ago

Pretty sure we have CI that run on msys, so it cannot be totally broken. Something must be difference in your case... Is the path C:/msys64/home/user.name/src/tmp/test.sh wrong? What if you run bash C:/msys64/home/user.name/src/tmp/test.sh HELLO manually in your msys terminal, does it work?

db-src commented 3 years ago

Thanks for looking. Yes, I did of course check beforehand that the path is correct, ls works, all variations of running the script directly and via bash that I could think up work, etc.

db-src commented 3 years ago

I can replicate this on a brand new machine and installation of MSYS2/MinGW64/Meson/etc. - with both my real project and the toy one supplied here. Thus it doesn't seem like anything I'm doing wrong. Can someone advise how I can ever use Meson on Windows/MSYS2 again, or...?

$ meson --prefix /tmp/install/test/ ~/src/test/
The Meson build system
Version: 0.58.1
Source dir: C:/msys64/home/User.Name/src/test
Build dir: C:/msys64/tmp/build/test
Build type: native build
Project name: test
Project version: undefined
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program test.sh found: YES (bash C:/msys64/home/User.Name/src/test/test.sh)
Build targets in project: 0

Found ninja-1.10.2 at C:\msys64\mingw64\bin/ninja.EXE

User.Name@COMP-NAME MINGW64 /tmp/build/test
$ ninja install
[0/1] Installing files.
Running custom install script 'bash C:/msys64/home/User.Name/src/test/test.sh'
--- stdout ---

--- stderr ---
/bin/bash: C:/msys64/home/User.Name/src/test/test.sh: No such file or directory

FAILED: install script 'bash C:/msys64/home/User.Name/src/test/test.sh' exit code 3221225477, stopped
FAILED: meson-install
"C:/msys64/mingw64/bin/meson" "install" "--no-rebuild"
ninja: build stopped: subcommand failed.
db-src commented 2 years ago

This is still broken on

Please, does anyone care and can put in some time to try to reproduce/help?

I now get a slightly different error:

$ ninja install
[0/1] Installing files.
Running custom install script 'bash C:/msys64/home/User.Name/src/tmp/test/test.sh'
--- stdout ---

--- stderr ---

FAILED: install script 'bash C:/msys64/home/User.Name/src/tmp/test/test.sh' exit code 3221225477, stopped
FAILED: meson-install
"C:/msys64/mingw64/bin/meson" "install" "--no-rebuild"
ninja: build stopped: subcommand failed.

Running said script/command directly works fine:

$ bash C:/msys64/home/User.Name/src/tmp/test/test.sh
hiya
$ echo $?
0

It's just that, again on two machines so far, ninja install is unusable with even the simplest possible project given/tested above.

I would really appreciate being able to use this again, maybe some time this year...

jpakkane commented 2 years ago 
meson.add_install_script('test.sh')

How are you specifying the HELLO argument there? It is not listed in the sample code.

db-src commented 2 years ago

Sorry, I had originally included the argument but have since removed it from my test scripts. As you can see, it still doesn't work without it, in any case.

jpakkane commented 2 years ago

Please copypaste the exact code you use as well as the entire error log.

db-src commented 2 years ago

The code is in the OP:

To Reproduce meson.build:

project('test')
meson.add_install_script('test.sh')

tesh.sh:

#!/bin/bash
echo hiya

Here is the full output showing the directory being cleaned, file contents verified, and what Meson and Ninja give me:

User.Name@HOST-NAME MINGW64 /tmp/build/test
$ rm -fr *

User.Name@HOST-NAME MINGW64 /tmp/build/test
$ cat ~/src/tmp/test/meson.build
project('test')
meson.add_install_script('test.sh')

User.Name@HOST-NAME MINGW64 /tmp/build/test
$ cat ~/src/tmp/test/test.sh
#!/bin/bash
echo hiya

User.Name@HOST-NAME MINGW64 /tmp/build/test
$ meson ~/src/tmp/test
The Meson build system
Version: 0.60.3
Source dir: C:/msys64/home/User.Name/src/tmp/test
Build dir: C:/msys64/tmp/build/test
Build type: native build
Project name: test
Project version: undefined
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program test.sh found: YES (bash C:/msys64/home/User.Name/src/tmp/test/test.sh)
Build targets in project: 0

Found ninja-1.10.2 at C:\msys64\mingw64\bin/ninja.EXE

User.Name@HOST-NAME MINGW64 /tmp/build/test
$ ninja install
[0/1] Installing files.
Running custom install script 'bash C:/msys64/home/User.Name/src/tmp/test/test.sh'
--- stdout ---

--- stderr ---

FAILED: install script 'bash C:/msys64/home/User.Name/src/tmp/test/test.sh' exit code 1, stopped
FAILED: meson-install
"C:/msys64/mingw64/bin/meson" "install" "--no-rebuild"
ninja: build stopped: subcommand failed.

User.Name@HOST-NAME MINGW64 /tmp/build/test
$

The install- and meson-log files don't show anything useful:

User.Name@HOST-NAME MINGW64 /tmp/build/test
$ ls meson-logs/
install-log.txt  meson-log.txt

User.Name@HOST-NAME MINGW64 /tmp/build/test
$ cat meson-logs/install-log.txt
# List of files installed by Meson
# Does not contain files installed by custom scripts.

User.Name@HOST-NAME MINGW64 /tmp/build/test
$ cat meson-logs/meson-log.txt
Build started at 2022-01-10T09:45:52.423722
Main binary: C:/msys64/mingw64/bin/python.exe
Build Options:
Python system: Windows
The Meson build system
Version: 0.60.3
Source dir: C:/msys64/home/User.Name/src/tmp/test
Build dir: C:/msys64/tmp/build/test
Build type: native build
Project name: test
Project version: undefined
Build machine cpu family: x86_64
Build machine cpu: x86_64
Host machine cpu family: x86_64
Host machine cpu: x86_64
Target machine cpu family: x86_64
Target machine cpu: x86_64
Program test.sh found: YES (bash C:/msys64/home/User.Name/src/tmp/test/test.sh)
Build targets in project: 0

Found ninja-1.10.2 at C:\msys64\mingw64\bin/ninja.EXE

User.Name@HOST-NAME MINGW64 /tmp/build/test
$

Do you need anything else? Thanks for looking.

jpakkane commented 2 years ago

MinGW has issues every now and then. Could you try a plain Python script instead? Put the following content in file iscript.py:

#!/usr/bin/env python3

print('Script is working')

And then replace your script command with:

meson.add_install_script('iscript.py')
db-src commented 2 years ago

Thanks for the pointer. That works OK:

User.Name@HOST-NAME MINGW64 /tmp/build/test
$ rm -fr *

User.Name@HOST-NAME MINGW64 /tmp/build/test
$ meson ~/src/tmp/test
The Meson build system
Version: 0.60.3
Source dir: C:/msys64/home/User.Name/src/tmp/test
Build dir: C:/msys64/tmp/build/test
Build type: native build
Project name: test
Project version: undefined
Host machine cpu family: x86_64
Host machine cpu: x86_64
Program iscript.py found: YES (C:/msys64/mingw64/bin/python.exe C:/msys64/home/User.Name/src/tmp/test/iscript.py)
Build targets in project: 0

Found ninja-1.10.2 at C:\msys64\mingw64\bin/ninja.EXE

User.Name@HOST-NAME MINGW64 /tmp/build/test
$ ninja install
[0/1] Installing files.
Running custom install script 'C:/msys64/mingw64/bin/python.exe C:/msys64/home/User.Name/src/tmp/test/iscript.py'

User.Name@HOST-NAME MINGW64 /tmp/build/test

So we know it's somehow to do with bash on MSYS2?

I'd hesitate to have to rewrite my current install scripts in Python instead, as they are heavily reliant on shell piping. Will it be able to handle that? My knowledge of Python is so slim that it could be a challenge anyway.

So if there's anything further I can do to help debug the bash issue, please let me know - happy to run all kinds of debug modes, traces, etc.!

eli-schwartz commented 2 years ago

Are you using the msys2 native package for meson or are you running a windows native version of meson?

In the latter case the calling conventions for the bash program might be getting confused between windows and msys2 invocation.

jpakkane commented 2 years ago

I'd hesitate to have to rewrite my current install scripts in Python instead, as they are heavily reliant on shell piping. Will it be able to handle that? My knowledge of Python is so slim that it could be a challenge anyway.

That is what I would do. Python is a lot more readable and performant than shell pipelines and more portable to boot (every unix userland behaves annoyingly differently).

eli-schwartz commented 2 years ago

and more portable to boot (every unix userland behaves annoyingly differently).

Rephrased:

Most people writing shell scripts have no clue what they are doing, and may not actually be aware POSIX exists -- they certainly don't try sticking to it, and even the rare few who do try sticking to it, usually fail because they don't actually know what is POSIX and what isn't. Also, Windows is not POSIX so using shell scripts automatically sacrifices windows native compatibility.

db-src commented 2 years ago

My bash scripts aren't doing anything too advanced I don't think, just some sed, cut, sort, uniq, etc... but I don't want to get into a holy war or anything ;-) It was just easiest to cobble together what I needed from a few common (and I thought fairly standard/POSIXy) commands than to learn how to do it in another language... If I have to rewrite, so be it, but I'd rather avoid.

Are you using the msys2 native package for meson or are you running a windows native version of meson?

In the latter case the calling conventions for the bash program might be getting confused between windows and msys2 invocation.

It's the MinGW64 package, so I guess that counts as Windows-native? Should I try msys/meson 0.60.2-3 instead?

User.Name@HOST-NAME MINGW64 /tmp
$ pacman -Ss meson
mingw32/mingw-w64-i686-meson 0.60.3-1
    High-productivity build system (mingw-w64)
mingw64/mingw-w64-x86_64-meson 0.60.3-1 [installed]
    High-productivity build system (mingw-w64)
ucrt64/mingw-w64-ucrt-x86_64-meson 0.60.3-1
    High-productivity build system (mingw-w64)
clang64/mingw-w64-clang-x86_64-meson 0.60.3-1
    High-productivity build system (mingw-w64)
msys/meson 0.60.2-3
    High-productivity build system

User.Name@HOST-NAME MINGW64 /tmp
$ which meson
/mingw64/bin/meson

User.Name@HOST-NAME MINGW64 /tmp
$
eli-schwartz commented 2 years ago

The mingw package is intended for using the msys2 ecosystem as a delivery platform for the getting windows-native toolchains installed to build windows-native binaries (that's what mingw is for after all).

The msys package is intended for using the msys2 ecosystem to build msys2 packages, linked to the msys runtime.

So yeah, you tried to use a windows native meson, essentially, but have it run an install script in bash with bash coming from the msys runtime. I am no expert on the topic, but it's certainly a case where I'd expect the potential for problems.

In particular, whether you can run the same command in the msys2 shell successfully is probably not indicative of anything. A better indicator of whether it should work is whether you'd be able to run the same command from Windows' built-in cmd.exe (since that is the same way meson tries to run it).

Using msys/meson would be far more likely to work with bash based install scripts. That being said, if your software is supposed to be built for windows distribution rather than only for msys2 (or Linux/BSD) you might well be advised to write the script in python instead.

db-src commented 2 years ago

If I use msys versions of Meson and Ninja instead, still in the MinGW64 shell (because it has the libraries I need, and I want/need to use its native compiler and other tools), I just get this:

$ meson ~/src/tmp/test
ERROR: This python3 seems to be msys/python on MSYS2 Windows, which is known to have path semantics incompatible with Meson
ERROR: Please install and use mingw-w64-i686-python3 and/or mingw-w64-x86_64-python3 with Pacman

So I'm guessing this combination can't possibly work?

yume-chan commented 8 months ago

I used Procmon (https://learn.microsoft.com/en-us/sysinternals/downloads/procmon) to trace process starts, and found python starts C:\WINDOWS\SYSTEM32\bash.exe (from WSL) instead of D:\msys64\usr\bin\bash.exe, even though the PATH environment variable includes D:\msys64\usr\bin\.

I think this is because system folder has higher priority than PATH variable, see https://learn.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createprocessa:

[...] If the file name does not contain a directory path, the system searches for the executable file in the following sequence:

  1. The 32-bit Windows system directory. Use the GetSystemDirectory function to get the path of this directory.

[...]

  1. The directories that are listed in the PATH environment variable. Note that this function does not search the per-application path specified by the App Paths registry key. To include this per-application path in the search sequence, use the ShellExecute function.

I don't know why msys2 bash can start the "correct" bash, maybe because it doesn't use CreateProcess directly, but compiled against Cygwin which does its own resolving?

Another question is where did meson translate meson.add_install_script('test.sh') to bash path/to/test.sh (I see the full command line is this). Is it possible for meson to use the full path of msys2 bash here?