npx does not recognize local binaries inside VirtualBox shared folders

[bndnico]

I have set up a development environment inside VirtualBox virtual machine. The host computer runs on Mac OS X El Capitan (10.11.6), the VM is Ubuntu 16.04 LTS. npx runs fine for file paths inside the VM (e.g. /home), but not in paths that are "shared folders", i.e. folders that reside on the host's file system and are mapped into the VM by VirualBox (https://www.virtualbox.org/manual/ch04.html#sharedfolders).

I know that symlinks are disabled inside shared folders by default (for security reasons, see https://askubuntu.com/a/446328) but I have configured the virtual machine accordingly, so symlinks now work. I don't understand what additional conditions cause npx to fail..

Steps to reproduce the issue:

1. VM-local path (/tmp)

~$ mkdir /tmp/cowtest
~$ cd /tmp/cowtest
/tmp/cowtest$ npm install -D cowsay
(...)
+ cowsay@1.3.0
added 10 packages from 3 contributors in 1.138s
/tmp/cowtest$ npx --no-install cowsay moo
 _____
< moo >
 -----
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

✅ Everything works as expected!

2. Shared folder (/media/sf_data)

/tmp/cowtest$ mkdir /media/sf_data/cowtest
/tmp/cowtest$ cd /media/sf_data/cowtest/
/media/sf_data/cowtest$ npm install -D cowsay
(...)
+ cowsay@1.3.0
added 10 packages from 3 contributors in 1.324s
/media/sf_data/cowtest$ npx --no-install cowsay moo
not found: cowsay

:x: Although I have executed exactly the same commands, npx only prints "not found" in the second case. (I added --no-install because otherwise, it would simply download the module again and again.)

However, if I run the cowsay command manually, it works fine:

/media/sf_privdev/cowtest$ ./node_modules/.bin/cowsay moo
 _____
< moo >
 -----
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Why doesn't it find the local binary in the scond case?

[zkat]

I believe this is a duplicate of https://github.com/zkat/npx/issues/104 -- so you just need to upgrade your npx version, probably.

[bndnico]

I sudo apt upgradeed my system, upgraded NPM using sudo npm install -g npm@latest. I then even installed npx separately, because I think the version shipped with NPM is much older: sudo npm install -g npx.

However, the problem is still there. Is there anything else I can update? Here is the log with all version information:

/media/sf_data/cowtest$ cat /etc/issue
Ubuntu 16.04.4 LTS \n \l

/media/sf_data/cowtest$ node -v
v8.11.1
/media/sf_data/cowtest$ npm -v
6.0.0
/media/sf_data/cowtest$ npx -v
10.2.0
/media/sf_data/cowtest$ npx --no-install cowsay moo
not found: cowsay
/media/sf_data/cowtest$ ./node_modules/.bin/cowsay moo
 _____
< moo >
 -----
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

[zkat]

whoa ok. This is definitely weird. Reopening and looking into it. O.o

[zkat]

@bndnico can you tell me what each of these commands prints for you?

• $ npx -c "echo $PATH"
• $ npx -c "which cowsay"
• $ npx which@latest cowsay; echo $? (this will use node-which, which is what npx uses for this, instead of system which)

[bndnico]

Sure!

/media/sf_data/cowtest$ npx -c "echo $PATH"
/home/nico/bin:/home/nico/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
/media/sf_data/cowtest$ npx -c "which cowsay"
/media/sf_data/cowtest/node_modules/.bin/cowsay
/media/sf_data/cowtest$ npx which@latest cowsay; echo $?
npx: installed 2 in 2.308s
1

Here is the output of the same commands from /tmp, where "npx cowsay" works as expected:

/tmp/cowtest$ npx -c "echo $PATH"
/home/nico/bin:/home/nico/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
/tmp/cowtest$ npx -c "which cowsay"
/tmp/cowtest/node_modules/.bin/cowsay
/tmp/cowtest$ npx which@latest cowsay; echo $?
npx: installed 2 in 1.845s
/tmp/cowtest/node_modules/.bin/cowsay
0

[zkat]

I think this might be a bug with node-which?

Try npm i which and then npx node -e "require('which')('cowsay', console.error)".

npx just throws away errors from it so it might just be ignoring an important error lol

[bndnico]

/media/sf_data/cowtest$ npx node -e "require('which')('cowsay', console.error)"
{ Error: not found: cowsay
    at getNotFoundError (/media/sf_data/cowtest/node_modules/which/which.js:13:12)
    at F (/media/sf_data/cowtest/node_modules/which/which.js:68:19)
    at E (/media/sf_data/cowtest/node_modules/which/which.js:80:29)
    at /media/sf_data/cowtest/node_modules/which/which.js:89:16
    at /media/sf_data/cowtest/node_modules/isexe/index.js:42:5
    at /media/sf_data/cowtest/node_modules/isexe/mode.js:8:5
    at FSReqWrap.oncomplete (fs.js:152:21) code: 'ENOENT' }

[zkat]

@bndnico this looks likes it's a bug in node-which. I don't really have time to look into it myself but you could try filing an issue on that repo -- I'd try to help more but I'm not sure what specifically is triggering this issue for you.

[bndnico]

Thanks for looking into this. I have analyzed the code and I have found that the problem is caused by the module "isexe" that is used by "which", specifically the file https://github.com/isaacs/isexe/blob/master/mode.js. It does not correctly recognize the program in "node_modules/.bin" as being executable because in the case of VirtualBox shared folders the file permissions are a bit special.

/media/sf_data/cowtest$ ls -la node_modules/.bin/cowsay
lrwxrwx--- 1 root vboxsf 16 Mai  5 11:41 node_modules/.bin/cowsay -> ../cowsay/cli.js
/media/sf_data/cowtest$ id
uid=1000(nico) gid=1000(nico) groups=1000(nico),4(adm),20(dialout),24(cdrom),27(sudo),30(dip),46(plugdev),113(lpadmin),128(sambashare),999(vboxsf)

As you can see, the file is only readable and executable by the user "root" and the group "vboxsf". My user is a member of "vboxsf", but it has it's own primary group. mode.js in the "isexe" module only checks if the primay group matches (process.getgid()) and ignores all secondary group memberships. That's why I can execute the program on the console and not via npx. I think, mode.js should use process.getgroups() instead of getgid().

I will file a bug, but unfortunately, the repository has not seen a lot of activity in the last few months...

[bndnico]

Just FYI: The issue I have created can be found at https://github.com/isaacs/isexe/issues/18. As expected, no reaction yet :-(