bhaible
commented
2 years ago A similar error occurs also when executing the configure script of https://ftp.gnu.org/gnu/coreutils/coreutils-9.1.tar.xz .
Open bhaible opened 2 years ago
bhaible
commented
2 years ago A similar error occurs also when executing the configure script of https://ftp.gnu.org/gnu/coreutils/coreutils-9.1.tar.xz .
angelhof
commented
2 years ago Thanks for reporting this! gnulib-tool seems to be a great stress test for PaSh.
I narrowed down the issue in gnulib-tool to the following function:
#!/bin/sh
func_emit_tests_Makefile_am ()
{
ofd=3
{
echo hi
} >&$ofd
}Based on shellcheck's output, this (>&$ofd) doesn't look like POSIX compliant syntax.
PaSh's parser only supports POSIX compliant syntax (even though it runs scripts on bash under the hood). I don't see an immediate path for PaSh to support non-POSIX compliant syntax (that would not require significant engineering) since some of its transformations are built on the structure of the AST that is returned by a POSIX compliant dash parser reimplementation.
After finding the parsing issue I replaced $ofd with 3 (which is OK w.r.t POSIX), and now PaSh manages to parse the file, but now there are some other failures (I fixed one #575 which had to do with some file descriptors being left open but I will continue investigating).
thesamesam
commented
2 years ago AFAIK most or all of gnulib is supposed to be compatible with POSIX shell so this should be fixable. Ditto configure scripts, fortunately!
bhaible
commented
2 years ago Thanks for trimming that down to a small test case; I would not have known how to do that.
Based on shellcheck's output, this (>&$ofd) doesn't look like POSIX compliant syntax.
I think it is POSIX compliant syntax, and shellcheck's warning is a bug: https://github.com/koalaman/shellcheck/issues/2520
Additionally, for this slightly modified shell script
#!/bin/sh
func_emit_tests_Makefile_am ()
{
ofd=3
{
echo hi
} 1>&$ofd
}
func_emit_tests_Makefile_am 3>&1for which pa.sh fails to parse in the same way, shellcheck does not give a warning.
angelhof
commented
2 years ago Thanks for checking this! By the way, I only mentioned shellcheck as evidence that this is not POSIX compliant (I am not sure if we can trust it for absolute precision, it probably has both false positives and false negatives).
We will investigate the parsing issue. At the same time, we will investigate the more important failures that happen once the script is parsed :)
bhaible
commented
2 years ago we will investigate the more important failures that happen once the script is parsed :)
When you get to this stage, you'll need a GNU gnulib checkout (see https://savannah.gnu.org/git/?group=gnulib ) and put the script into the top-level directory of that checkout (so that it finds the various input files).
mgree
commented
2 years ago I can confirm that the POSIX spec allows this behavior, and dash accepts it:
$ fd=2
$ echo hi >&$fd
hi
$ fd=3
$ echo hi >&$fd
/Users/mgree/smoosh/libdash/src/dash: 4: 3: Bad file descriptorSmoosh works as well, using the OCaml version of our bindings:
$ fd=2
$ echo hi >&$fd
hi
$ fd=3
$ echo hi >&$fd
Bad file descriptor: 3So I suspect there's a bug somewhere in our Python bindings or in our handling of the resulting AST.
angelhof
commented
2 years ago Thanks @mgree, I fixed it in this PR ( #582 ) actually! This was an untested corner of the parser and it needed one extra dereference.
There are still issues with the gnulib-tool which require investigation but parsing seems to pass now.
gnulib-toolis a large shell script, from the GNU gnulib project. When I attempt to execute it through pash, with specific arguments, there is a parse error.Here is gnulib-tool-cmp, the gnulib-tool script with an extra line 45 that sets the arguments. For successful execution, you would also need a gnulib git checkout. But pa.sh fails to parse the script, even before it gets to reading other files.
gnulib-tool-cmd.gz
This is on an Ubuntu 22.04 machine.