[Bug] dbux is assuming `bash` in Windows resolves to MINGW/MSYS bash

[CherryDT]

Bug Report

Bug Summary dbux seems to assume that running bash in Windows will start some sort of MINGW/MSYS bash such as Git bash. However, by default, the only bash.exe in Windows will be the wrapper that launches /bin/bash in the default WSL distro (if WSL is enabled).

Steps To Reproduce

1. Make sure WSL is installed
2. Open some JavaScript file and click "Start DBux"
3. See error in terminal

Expected behavior dbux should either not require a specific kind of bash installed at all or tell the user exactly what is expected and ask for its location if it's not in PATH.

Actual behavior dbux launches bash which by default will by C:\Windows\System32\bash.exe (a helper to launch bash inside WSL). It then fails with a misleading file-not-found error message because it launches node from the shell which is now inside WSL and passes a Windows-style path to it.

The loading box [Dbux] : New version. Installing 1 library/ies (1-3 mins)... then stays there forever.

Attachments

Screenshots

System (please complete the following information):

• OS: Windows 10
• Dbux VSCode Extension version: 0.6.15

(I updated the bug description based on what the actual problem is, because my initial assumption was wrong.)

[Domiii]

Thank you for the analysis. Very helpful!

I don't think, using VSCode's internal copy of node is a good idea because that is usually not maintained or controllable by you. Instead, what I am going to do is: make the path configurable, so, in edge cases like yours, you can just tell it which version you want. Would that work for you?

NOTE: I have not used WSL before.

PS: For future reference, would be great if you can also provide the Dbux Output channel log.

[CherryDT]

Ah I see what you mean (I didn't realize it's what's used for running the code too, I thought it was just part of the installation process), yes if it's configurable it should be fine.

Output channel log: sorry I (apparently wrongly) thought you meant what was already in the terminal.

[Domiii]

@CherryDT I just deployed the fix. I hope it works. We did not do a good job organizing system dependencies. Had to find them, and I might have overlooked a few.

I recommend the following:

1. Upgrade to latest. Make sure you are at least on 0.6.15 (sometimes VSCode needs a little while to make the new version the default, but you can choose it manually from the gear icon on the extension button, inside of the VSCode extensions list)
2. Change all the dbux.paths.X settings, as you need, especially bash, node and npm. Others, if you want to play with the Dbux Practice (i.e. "tutorial-ish") exercises.
3. Possibly restart.
4. And go!
5. Once installed (once "Dbux Start") has succeeded, you can also run the Check System command:

If everything is successful, from the Practice view, feel free to try some of the pre-prepared practice exercises.

Let me know, if there are more problems!

[CherryDT]

Thank you!

I just realized this probably (?) has nothing to do with my terminal shell being configured to WSL bash in particular, as I first assumed - instead it seems you simply invoke bash, which will be the Linux bash if WSL is installed on the system (which, in my opinion, is no edge case for a JavaScript developer nowadays).

I fixed the issue by changing the new bash path option to C:\Program Files\Git\bin\bash.exe.

(By the way: In a WSL window, running DBux works fine.)

[CherryDT]

Another thing I noticed: I saw a connection error in the terminal, but dbux then apparently issued another (different) command even though the first one failed - shouldn't this abort the operation entirely?

[Dbux run.js] (Terminal task running for 590.59 seconds.)
[Dbux run.js] (Terminal task running for 600.59 seconds.)
[Dbux run.js] (Terminal task running for 610.60 seconds.)
[Dbux run.js] (Terminal task running for 620.61 seconds.)
npm ERR! code ECONNABORTED
npm ERR! syscall connect
npm ERR! errno ECONNABORTED
npm ERR! request to https://registry.npmjs.org/@datastructures-js%2fpriority-queue failed, reason: connect ECONNABORTED 104.16.26.35:443

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users\david\AppData\Local\npm-cache\_logs\2022-03-25T16_02_49_895Z-debug-0.log

david@CHE-T14S MINGW64 ~/.vscode/extensions/domi.dbux-code-0.6.15
$ "node" "C:/Users/david/.vscode/extensions/domi.dbux-code-0.6.15/resources/src/_dbux_run.js" 1 eyJjd2QiOiJDOi9Vc2Vycy9kYXZpZC8udnNjb2RlL2V4dGVuc2lvbnMvZG9taS5kYnV4LWNvZGUtMC42LjE1IiwiY29tbWFuZCI6IlwiQzpcXFByb2dyYW0gRmlsZXNcXG5vZGVqc1xcbnBtLmNtZFwiIGkgQGRidXgvY2xpQDAuNi4xNSIsInRtcEZvbGRlciI6IlI6L1RlbXAvZGJ1eC01S29pNDMiLCJzaGVsbCI6IkM6XFxQcm9ncmFtIEZpbGVzXFxHaXRcXGJpblxcYmFzaC5leGUifQ==
[Dbux run.js] command received: {
  nodePath: 'C:\\Program Files\\nodejs\\node.exe',
  cwd: 'C:/Users/david/.vscode/extensions/domi.dbux-code-0.6.15',
  command: '"C:\\Program Files\\nodejs\\npm.cmd" i @dbux/cli@0.6.15',
  tmpFolder: 'R:/Temp/dbux-5Koi43',
  shell: 'C:\\Program Files\\Git\\bin\\bash.exe'
}
[Dbux run.js] (Terminal task running for 10.00 seconds.)
[Dbux run.js] (Terminal task running for 20.01 seconds.)
[Dbux run.js] (Terminal task running for 30.01 seconds.)
[Dbux run.js] (Terminal task running for 40.03 seconds.)
[Dbux run.js] (Terminal task running for 50.03 seconds.)
[Dbux run.js] (Terminal task running for 60.04 seconds.)
[Dbux run.js] (Terminal task running for 70.04 seconds.)

Also, I noticed when I click "Start DBux" it goes to a screen that looks like it already started, even when it's not ready yet. If the commands then fail, the only way to go back to the previous screen with "Start DBux" on it is to reload the window...

[CherryDT]

Update: Actually, I realized now, it does not work correctly in a WSL window, because now it seems to use the custom path I set in the regular VSCode window also for the WSL window, so while it initally uses the right bash it then passes the Windows path that I set into the script...

Something doesn't make sense here - in the WSL tab of the settings it shows bash but "modified in: User", while in the User tab it shows my Windows path - however in a regular VSCode window that User tab is the only place I can change it, so I'm stuck again...

david@CHE-T14S:~/.vscode-server/extensions/domi.dbux-code-0.6.15 $ "node" "/home/david/.vscode-server/extensions/domi.dbux-code-0.6.15/resources/src/_dbux_run.js" 1 eyJjd2QiOiIvaG9tZS9kYXZpZC8udnNjb2RlLXNlcnZlci9leHRlbnNpb25zL2RvbWkuZGJ1eC1jb2RlLTAuNi4xNSIsImNvbW1hbmQiOiJcIm5wbVwiIGluc3RhbGwgLS1vbmx5PXByb2QiLCJ0bXBGb2xkZXIiOiIvdG1wL2RidXgtNVNIMnpTIiwic2hlbGwiOiJDOlxcUHJvZ3JhbSBGaWxlc1xcR2l0XFxiaW5cXGJhc2guZXhlIn0=
[Dbux run.js] command received: {
  nodePath: '/home/david/.asdf/installs/nodejs/16.10.0/bin/node',
  cwd: '/home/david/.vscode-server/extensions/domi.dbux-code-0.6.15',
  command: '"npm" install --only=prod',
  tmpFolder: '/tmp/dbux-5SH2zS',
  shell: 'C:\\Program Files\\Git\\bin\\bash.exe'
}
Error: Error: spawn C:\Program Files\Git\bin\bash.exe ENOENT
    at Process.ChildProcess._handle.onexit (node:internal/child_process:282:19)
    at onErrorNT (node:internal/child_process:477:16)
    at processTicksAndRejections (node:internal/process/task_queues:83:21) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'spawn C:\\Program Files\\Git\\bin\\bash.exe',
  path: 'C:\\Program Files\\Git\\bin\\bash.exe',
  spawnargs: [ '-c', '"npm" install --only=prod' ]
}

How can I make it work correctly in both regular and WSL windows in VSCode?

[Domiii]

Thanks again for three more bug reports! Much appreciated.

1. I saw a connection error in the terminal, but dbux then apparently issued another (different) command even though the first one failed - shouldn't this abort the operation entirely?

   • I am not sure what is going on there. Some commands allow the terminal window to be re-used, so the second command might or might not have anything to do with the first one. I would need to see the whole log. Can you cut the repeating "task running" messages out and post the rest of the log?
   • It looks like this is happening after you clicked the "Start Dbux" button. Is that correct?

2. I noticed when I click "Start DBux" it goes to a screen that looks like it already started, even when it's not ready yet.

   • I am not sure what that means. Maybe you can produce a screenshot or a short gif of the phenomenon?

3. in the WSL tab of the settings it shows bash but "modified in: User", while in the User tab it shows my Windows path

   • According to this, this issue appears to generally be part of the much bigger problem of getting advanced extensions to work correctly in WSL, or other remote environments.

To continue point (3):

In fact, your original problem looks like it is not the wrong executable paths, but rather the fact that your WSL environment cannot pick up files from the locally installed extension. Meaning, you want to keep your executable paths to node, npm and bash, and instead, you want to somehow map the locally prepared _dbux_run.js asset to a location where your WSL environment can pick it up.

What I mean is: Check out your original screenshot. You see that it cannot find _dbux_run.js, and that is because, what I am guessing, Dbux and its assets are installed locally, and so your remote node cannot pick it up.

The documentation on how to get extension working in remote environments still seems very immature. However, they vaguely hint at the fact that it can be useful to just override an extension's execution location here.

So here we go. Try the following:

1. Unset all the dbux.paths.
   • Again: this is because, you don't want to run windows executables on the remote.
2. Follow that guide, and try to force Dbux to run in workspace? That should be as simple as adding the following to your User Settings JSON file:

   "remote.extensionKind": {
      "Domi.dbux-code": ["workspace"]
   }

This way, it should be able to find the _dbux_run.js file (in theory anyway).

Let me how badly it goes? That sad state of the so-called "documentation" of how to make extensions run in remote environments makes me somewhat pessimistic 🤣

[Domiii]

@CherryDT

If the above does not work: good news! You do not need to run the Dbux extension in WSL at all. Instead:

1. Just run your application with Dbux enabled on the remote.
   • Specifically, if your application runs on Node, use the @dbux/cli manually.
2. Then run the Dbux extension locally (on Windows).
3. Now, I need to make the runtime server address configurable. Then, when you run the application, you tell Dbux the correct address, and it should work fine.
4. Finally, for this to work, you have to make sure you can access Windows ports from within WSL. Not sure how that works. Again, I have no experience with WSL.

[CherryDT]

I am not sure what is going on there. Some commands allow the terminal window to be re-used, so the second command might or might not have anything to do with the first one. I would need to see the whole log. Can you cut the repeating "task running" messages out and post the rest of the log?

I have a hard time reproducing it now (it happened when my Internet connection was flaky so I got some connection errors - but for some reason when I now just disconnect my Internet to simulate it, all that happens is that npm gets stuck...?) - but it seems that normally, the commands npm i --only=prod and npm i @dbux/cli are run in order, and what I meant was that I noticed that the npm i @dbux/cli command was executed even when the npm i --only=prod command failed (had exit code other than zero), and I was thinking that probably the whole chain should stop in that case.

It looks like this is happening after you clicked the "Start Dbux" button. Is that correct? Yes.

I am not sure what that means. Maybe you can produce a screenshot or a short gif of the phenomenon? What I mean is this:

Here I did finally get it to crash by messing with my Internet, but only during the second command and not the first as last time. The result is that the loader box in the bottom right just stays there forever, and in the sidebar I get the UI of a dbux-that's-ready, which in fact I already get immediately after clicking "start dbux". Of course all the buttons there are non-functional now, but still, the UI of before ("start dbux") is gone and I can only retry the installation by reloading the window.

In fact, your original problem looks like it is not the wrong executable paths, but rather the fact that your WSL environment cannot pick up files from the locally installed extension. Meaning, you want to keep your executable paths to node, npm and bash, and instead, you want to somehow map the locally prepared _dbux_run.js asset to a location where your WSL environment can pick it up.

No I think there was some misunderstanding. I originally thought it had to do with my default shell being set to WSL bash even in Windows VSCode windows but that was a mistake, I realized afterwards that I had this on my old PC but on this one I have it on the stock setting which is PowerShell, since I use WSL remote 95% of the time anyway. The issue was that dbux launched bash and expected to get a shell that understands Windows paths (albeit with forward slashes), as Git bash would, but it failed because it got Ubuntu's bash in WSL instead (which is the default scenario if you set up WSL, because there will be a bash.exe in C:\Windows\System32 which launches bash in the default WSL distro). This was fixed by you adding those path configs and me setting it to C:\Program Files\Git\bin\bash.exe. This then however caused the opposite problem, where now in a WSL remote window it would attempt to launch C:\Program Files\Git\bin\bash.exe (which Linux wouldn't understand) instead of /bin/bash (here, just bash would work). This is not a problem for node/npm because I have node set up both in Windows and in WSL and just launching node would launch the correct one depending on the execution environment.

The only reason _dbux_run.js was "not found" is because the Windows path C:/Users/david/.vscode/extensions/domi.dbux-code-0.6.13/resources/src/_dbux_run.js was passed to Linux' bash, and in term, Linux' node (due to the wrong bash being executed in that scenario as explained above). bash Linux node treated it as a path relative to the current directory (which was /mnt/z/.vscode/extensions/domi.dbux-code-0.6.13), resulting in the non-existing path /mnt/z/.vscode/extensions/domi.dbux-code-0.6.13/C:/Users/david/.vscode/extensions/domi.dbux-code-0.6.13/resources/src/_dbux_run.js, as you can see in the error message's path.

(Note: I have a drive Z:\ that is symlinked on an NT object manager level to C:\Users\david, so here the /mnt/z and Z:\ and C:\Users\david are all the same, the reason you have two different variants at the same time is because some Windows API functions that resolve paths to their absolute versions do turn Z:\ into C:\Users\david, but that doesn't matter here at all.)

You see that it cannot find _dbux_run.js, and that is because, what I am guessing, Dbux and its assets are installed locally, and so your remote node cannot pick it up.

I see why you would think that, but that's not the case because a) the original screenshot was done from a regular Windows VSCode window, not a WSL remote window (it failed because it launched Ubuntu's bash in the process instead of Git bash), and b) if it were run from a WSL remote window, it would use its own local copy of the extension inside WSL (I did install the extension in both Window VSCode and WSL remote VSCode environments).

You do not need to run the Dbux extension in WSL at all. Instead: [...]

Well, maybe I don't need to, but this other solution sounds a heck of a lot more complicated. For starters, I'd now have to have two VSCode windows open, the WSL remote window I use for debugging but then an additional regular VSCode window for running the extension in...? And then I would probably also run into file locking issues and such because I'd have to open the same files in both, I presume, which is normally something I'd avoid.

My workaround: I have found a solution now, but it's a hack. I created a file dbux-bash.cmd with the contents @"C:\Program Files\Git\bin\bash" %* in a folder that's in my Windows PATH, and I've also created a symlink from ~/bin/dbux-bash.cmd to /bin/bash in WSL. Now it all works because on Windows, dbux will launch dbux-bash.cmd which will execute this one-liner batch script that ends up calling Git bash, while in a WSL remote window, it will end up calling ~/bin/dbux-bash.cmd which is nothing other than /bin/bash. :)

It all works beautifully this way!

The only thing needed to make this not a hack anymore would be a way to set the paths independently depending on environment (native, remote WSL, remote Docker, etc.) - it already seems to work to have Windows on the default of bash and change it for WSL, but what I need would be the opposite, or rather, allowing them to have independent settings from each other entirely.

[Domiii]

hi @CherryDT,

First of all, I'm glad you are saying that it works! That's awesome!

I see two more issues:

1. in the sidebar I get the UI of a dbux-that's-ready, which in fact I already get immediately after clicking "start dbux"

   • Maybe @MiccWan can take a look at that?
   • I think our installation process does not properly handle errors. Can we make sure that if any of the install commands fail, it fails gracefully? And also that it won't progress any further?
   • Maybe we can test it by just throwing errors from the install function on purpose?

2. a way to set the paths independently depending on environment (native, remote WSL, remote Docker, etc.)

   • Again, I am not too familiar with those remote environments, so I need a bit of help:
   • You have one VSCode instance that you can somehow run in local and in remote mode? Or are you concerned about different VSCode instances and Settings Sync, or...?
   • Can you give me an example of a command or an extension that can discern between local and remote env?

[CherryDT]

You have one VSCode instance that you can somehow run in local and in remote mode? Or are you concerned about different VSCode instances and Settings Sync, or...?

The former. There is one VSCode, but I have the Remote - WSL extension installed. I can then use the button in the bottom left to open a new window in a remote session, or launch it from WSL with code . etc. in the first place. I then get a window that is still Windows VSCode, but all access to files, etc. is handled internally by communicating with some kind of server that VSCode launches in the WSL environment, and some extensions (I guess that is some configuration inside the extension itself) also need to be installed separately in the WSL environment (you then get a button "Install in Remote WSL" instead of "Install", even if it's already installed in Windows) - some extensions just work out of the box (I guess those affect only the editor UI).

From what I experienced so far, configurations are sometimes shared between environments by default, not sure if it's always the case though. In the remote session's VSCode window, if you go to your settings, you'll get an extra tab "Remote: WSL" between "User" and "Workspace", and you can set different settings in the remote setting if you wish. I hardly ever had to do this though because most things just work great out of the box, and what I didn't understand in this case was why it seemed the priority while being in WSL was Default<Remote WSL<User<Workspace and not Default<User<Remote WSL<Workspace as I'd expect, because that now creates the situation where you can only have the Windows setting equal to the default while having the WSL setting different, but not the other way round (neither two different non-default values, of course).

Aaaah! I figured it out (while checking how it behaves with other extensions)! So, it appears, the thing that got me confused was how defaults are handled in VSCode's settings UI. There is no way in the UI to say "set that setting back to its default even though I modified it in another settings file further down the priority chain", and since bash was the default and I wanted that default changed in "User" but set back to what it was in "Remote: WSL", I couldn't make it work - but now that I realized it, I discovered two ways to make it work properly without turning to my hack from before:

1) It works if I use /bin/bash in the WSL-specific settings for dbux - same effect, but the string /bin/bash is different from the default bash so it's seen as override and works. 2) It also works if I manually put "dbux.paths.bash": "bash" in the remote's settings.json file. The issue is that if I set bash in the UI, the effect is that the UI removes the setting from the file, making it "use the default / the next lower-prioritized setting value instead".

So, the issue here is that the VSCode settings UI has no way to say "I want this value literally, even if it's the extension's default, and I don't want it to remove the setting from the file entirely because of that".

From your side, however, the best way would be to solve the issue that started all this - which is that on any Windows install with WSL set up so that WSL's bash is highest priority in the PATH (which is the default even with Git bash installed) things fail because you are running a true Linux shell instead of a MINGW-style shell like Git bash that supports Windows paths too - in such a way that you either don't require any specific shell at all, letting the system defaults do their job (I think that's what most other extensions do), or you do require MINGW bash on Windows (for example because you need more complex commands that are not portable across shells) but in that case you should probably better bundle it and use your own bundled version of it, in my opinion. (What if there is no MINGW bash installed at all? For example, if I don't use Git in Windows at all because all my development is in WSL and Git in Ubuntu suffices, then you will not even have any suitable bash.exe anywhere on the system.)

[Domiii]

hi @CherryDT,

1. don't require any specific shell at all, letting the system defaults do their job

   • We can't do that, since we need to run certain commands with some basic syntax that is just not the same across all shells.
   • Specifically: bash is just a system dependency that Dbux requires you to have, at least for now.

2. better bundle it

   • My basic rule is: don't bundle system executables, since it would bring much much pain to everyone involved.
   • If you don't want to install it, maybe you just can't use Dbux for now, or at least you cannot use it on the systems where you don't install it.

3. things fail because you are running a true Linux shell instead of a MINGW-style

   • I am still not quite sure what the real issue is here. It does not matter what kind of bash it is. The bash part works, as you have seen yourself.
   • The part that does not work is where WSL somehow mixes local vs. remote executables (e.g. bash or node) and local vs. remote file paths (i.e. the _dbux_run.js file that we use as a hackfix IPC bridge with the terminal process, as explained here).
   • This leads me to believe that the problem you are seeing is 99% caused by WSL. I am sure we can fix it in a better way, but for that I would need to understand, what actually gets messed up.
   • My guess is that in a WSL terminal, node x.js is actually changed entirely (by WSL), and instead executes node /remote/disk/x.js using the remote node executable. The problem here is then that that will simply not work if x.js is an absolute path. Which is ridiculous, since this would mean that the terminal simply does not support absolute paths, or in order to support absolute paths, you have to abide by unwritten/undocumented rules. Or maybe I am just ignorant, and there is a simple ruleset that one has to abide by, or other solution. Let us know, if you can find out how to get WSL to not override file paths (or at least override it in a predictable way).

[CherryDT]

I am still not quite sure what the real issue is here. It does not matter what kind of bash it is. The bash part works, as you have seen yourself.

No, the what I've seen is that dbux in Windows will work when talking to MINGW bash, and dbux in Ubuntu (WSL) will work then talking to Ubuntu's bash, because in both these cases node executable will be native to each system and the paths (either C:/stuff or /mnt/c/stuff) will be understood.

The issue is however that dbux in Windows assumes that the global command bash will resolve to a MINGW bash, which isn't the case by default when WSL is installed because Microsoft has a c:\windows\system32\bash.exe helper tool that will launch /bin/bash in the default WSL distribution. You are now running a Linux shell in (for all intents and purposes) a Linux system, albeit with some interop capabilities, so running node in there (not node.exe) will get you a node.js binary in Ubuntu and you won't have much luck with C:/stuff either.

The part that does not work is where WSL somehow mixes local vs. remote executables (e.g. bash or node) and local vs. remote file paths (i.e. the _dbux_run.js file that we use as a hackfix IPC bridge with the terminal process, as explained here).

This is not about remote windows. Again, remote works fine, because there you are running Linux stuff in Linux with Linux arguments and everything is fine. Instead, on a Windows window, you start a shell inside WSL, and that is the problem. If you would do ssh somelinuxserver.com -- bash instead, you'd also not expect it to work if you then pass node C:/stuff to it, no?

This leads me to believe that the problem you are seeing is 99% caused by WSL. I am sure we can fix it in a better way, but for that I would need to understand, what actually gets messed up.

Yes, this is what I meant - the fact that WSL is installed means that bash by default will resolve to C:\Windows\System32\bash.exe, which would launch a true Linux shell in a Linux environment, and that is unexpected for dbux and will deliver broken results if treated as if you were still in a Windows environment at that point.

My guess is that in a WSL terminal, node x.js is actually changed entirely (by WSL), and instead executes node /remote/disk/x.js using the remote node executable.

No, this is about the case of a Windows terminal unexpectedly starting a WSL bash, not a terminal in a WSL remote session. If you run node in Windows, you will get Windows' node. If you run node in Linux' bash, you'll get Linux' node, by the regular rules of how PATH is handled in Linux. The issue is that running bash from Windows will start a bash in Linux and not Windows so from that point onwards you are operating in Linux.

(If you'd run node.exe - with .exe - in Linux' bash however, you'd get Windows' node through interop because part of Windows' PATH is auto-appended to Linux' PATH and the WSL kernel supports a binfmt for Windows PE executables to forward the process to Windows, but there is no point in programmatically using a WSL shell in the middle anyway if you want a Windows thing to talk to other Windows things, so this isn't desired here.)

There is no magic happening with x.js though, because if you start bash and run node x.js, you literally just do that - pass the string node x.js to bash.

The problem here is then that that will simply not work if x.js is an absolute path.

True but only because you start an actual Linux bash instead of a bash designed for Windows and then feed it paths in Windows' format. (In the WSL remote window where you pass Linux paths, it all works fine.)

Which is ridiculous, since this would mean that the terminal simply does not support absolute paths, or in order to support absolute paths, you have to abide by unwritten/undocumented rules.

There is the wslpath utility as a helper to convert paths between Windows and Linux but you shouldn't need it, because as I said, in Windows everything would be fine if you'd use a bash designed for Windows, and in Linux you use a Linux bash.

Or maybe I am just ignorant, and there is a simple ruleset that one has to abide by, or other solution. Let us know, if you can find out how to get WSL to not override file paths (or at least override it in a predictable way).

I'm not sure in what way you see any file paths being overridden here.

The way I see it, it's simply like calling the wrong number. The one and only problem I see here is that with WSL installed on the machine, executing bash will give you a bash that runs in Linux and not Windows, which you don't expect and keep feeding it Windows things. That's why I suggested in some way making sure that you get a shell that's compatible with dbux (such as a MINGW/MSYS shell) by bundling it, or maybe downloading it on first run.

You said you don't want to do that, okay, but then there should probably be some setup process that makes sure the path setting for bash points to a compatible one. You could for example run bash --version and check what you get - with WSL you'll get something like GNU bash, version 5.0.17(1)-release (x86_64-pc-linux-gnu) while Git-on-Windows's bash will be like GNU bash, version 4.4.23(1)-release (x86_64-pc-msys) (you can see it say here msys and not linux-gnu). And instead of running Linux's bash and failing in mysterious ways, you could then ask the user to configure the "right" bash for Windows (and explain what sort of bash that would be). Or maybe you could do the more complex commands in a node script instead of bash's features for it, so you wouldn't have this dependency at all?

[Domiii]

• The issue is however that dbux in Windows assumes that the global command bash will resolve to a MINGW bash.

  • Dbux does not assume anything. It just runs bash. It cannot foresse that there is a clash between different bashs. That is either a VSCode issue, or there is some solution specific to VSCode+WSL interop, but so far I would not know what the solution is. Would have to dig deeper on that, if there is more people who desire for WSL support to improve.

• Or maybe you could do the more complex commands in a node script instead of bash's features for it, so you wouldn't have this dependency at all?

  • If you have a script that works, maybe provide it here, with some short explanations, and we leave it as documentation for the WSL crowd and link from the FAQ to here, for now.
  • I would suggest that we integrate those solutions into Dbux (and dedicate resources to testing etc.) once/if the WSL crowd swells to critical mass. But probably not for now, especially since the workaround kinda works :)

[CherryDT]

Dbux does not assume anything. It just runs bash. It cannot foresse that there is a clash between different bashs.

You run bash. You expect to get a MINGW/MSYS bash, behaving in the way your code assumes it does. That's an assumption. (not likely, but it could even be something else entirely that just happens to be called "bash" but isn't even a shell.)

By default, Windows either has no bash at all or it has WSL's bash if you enable that component. Anything else is a custom third party install that may or may not be what you want. I think it's kinda backwards to assume that the non-default thing that you want exists (especially since this is nowhere explained) while the closest thing to a default that you can get with that name on a Windows installation (bash in WSL) does not work with your code.

This has nothing to do with VSCode. It's just that MINGW/MSYS bash isn't a component commonly installed in Windows except as part of Git (but then unlike git.exe itself, bash.exe is not always in PATH). And if you Google bash for Windows you'll find that actually the Microsoft component called "bash on windows" is WSL... which won't work as you expect.

I'm not sure how else to describe this... I just think there are a few tiny pieces missing to make this a smooth experience.

(I mean... If you go to Microsoft's development evaluation VM download page and get a VM, install VSCode and the extension there and try it, you'll run into this exact problem...)

If you have a script that works, maybe provide it here, with some short explanations, and we leave it as documentation for the WSL crowd and link from the FAQ to here, for now.

Can you tell me what the commands are that you invoke in bash which need specific bash semantics, or point me to the locations in your code where you do that? Then I could create node scripts that do the same to replace the bash dependency. (Assuming I don't misunderstand you now.)

[Domiii]

• I'm not sure how else to describe this... I just think there are a few tiny pieces missing to make this a smooth experience.

  • No, I understand your point. WSL's bash for some reason won't execute the command right. (I'm not sure why, but that's what I gather from your analysis and the info you shared)
  • And I agree, things can always be a bit smoother :)

• Can you tell me what the commands are that you invoke in bash which need specific bash semantics, or point me to the locations in your code where you do that?

  • If you go to the Dbux output channel (there is even a command to take you there), search for wrapping terminal command. It should be mostly that.
  • There, it also shows you pathToDbuxRun, which from your report, seems to be the actual problem: the WSL bash (or the node executable that is invoked) is unable to find the path to the _dbux_run.js, i.e. something/something/dbux-code/resources/src/_dbux_run.js.

Thank you for your offer to look into it! Some of the things, I think you can do:

Maybe you can open a WSL bash manually (and use its corresponding node) and...

1. See what you need to do to get it to find and run that _dbux_run.js file?
   • (If you look at your original screenshot, that's what it complains about with a Cannot find module error message)
2. The Run Button actually just wraps @dbux/cli. Maybe you can also use that bash to just manually run @dbux/cli and see if that works?

[CherryDT]

Look, as I said before, trying to make dbux in Windows work with bash in Linux can't be the right choice. Not finding some file is just the tip of the iceberg. Don't forget you would be running stuff in an entirely different OS then! It would mean that suddenly the user would have to have node (in the correct version) installed in Linux even though they want to debug something on Windows, and starting the debugging would then also fail if any dependency is a native module (for example bcrypt) because the user might have run npm install in Windows which would have compiled Windows libraries for the native modules, but when using dbux it would suddenly attempt to run the code from Linux, failing to load the Linux library for the native module which was never compiled. Or the code that's debugged may use Windows-specific things and may just never be intended to run in another OS at all.

Bottom line is, when I have a regular VSCode window open in Windows, and I have some code there that I want to debug with dbux, I expect the code to be run and debugged in my regular OS, Windows, just like when I'd use the regular VSCode debugger. (And when I have a remote WSL window open, I'd expect my code to be run in Linux.) I certainly do not expect my Windows code to be run in Linux when I debug it.

So I think you should stop focusing on how you could somehow make it work to run stuff in the shell that belongs to a different operating system but instead we should look for a way to not even attempt that but use a regular shell that runs in Windows itself or remove the dependency on the particular shell entirely, by running everything in node as I suggested. That's what I wanted to help with.

Since you came back to talking about WSL not finding a file, I feel maybe there is still some misunderstanding here though? I mean, the reason the file wasn't found is just that you are starting a shell from another OS, then run that other OS' node binary, and try to pass a path from your own OS to the other OS' node's command line. It's quite easy to solve this particular issue with the path in fact but it would be a step in the wrong direction because we should be running anything in a different OS in the first place, because as explained above, there are more things that will then start to break!

---

Anyway, looking at the commands that are being run, I don't see why they would require bash in particular, can you elaborate?

I'm referring to this:

We can't do that, since we need to run certain commands with some basic syntax that is just not the same across all shells.

My goal was to do whatever requires that syntax inside a node script instead, and just invoke that script from the shell, which should have the same syntax in all shells.

[Domiii]

Hum, Ok... Maybe I misunderstood then. Again, I have never used WSL, and I also didn't quite know when you encounter what problem, what the circumstances are etc.

So to summarize what I understand the problem is:

When running a command, and we enforce a singular shell for all scenarios, things can go wrong because, the ideal shell actually depends on which mode VSCode is running in (seemingly, VSCode (or parts of it) seem to be able to run either in WSL or locally).

So to explain why we use bash: simply because we run a whole bunch of different commands in different scenarios, and having to support all kinds of different shells felt nightmarish, so I just wanted to ensure that we use a singular shell for everyone.

Some examples of where different shells work differently:

1. Some people have spaces in their executable paths (e.g. let's say you set your nodePath to point to c:/this is a valid path/node.exe), most shells would just want you to wrap the executable in double quotation marks, but PS for some reason doesn't work like that.
2. I don't quite remember, but I think the --flag syntax was the same in PS, bash and cmd.
3. I also vaguely remember that the the double dash operator does not always work as expected in PS.
4. The amount of ^ needed to escape a ^ varies between shells. Some people think that it is a package manager problem, but it turns out it is actually a nightmarish shell problem.
5. I'm sure there are more issues that I forgot...

I don't want to spend the time to find out how to get every command to work with PS, bash and cmd... and even more so: I don't want to consider all other possible shells that exist in the universe. That is why we are enforcing shell to be bash for now.

Possible Solutions

• I think, the easier way to move forward here is to somehow get VSCode to give us the "correct bash path" given the context. I.e., in a local VSCode instance, find out what we need to do to give us the correct bash on the local system. Conversely, in a WSL instance (is that how it works?), get VSCode (or the system or whoever) to give us the WSL bash instead. What do you think?
  • One possible issue is our use of shelljs's which.I just realized that in some places we look up the absolute path with that. Maybe which('bash') is one of the culprits involved here. You can help verify this by (for one) checking whether shellPath (which is kinda like which(cfg.dbux.paths.bash || 'bash')) is correct, when looking at the terminal debug info or the Dbux output window?
  • We use it for all our path lookup needs. If you use the Dbux: Check System Dependencies command (F1 or CTRL+P), you can also check if those paths make sense. If these paths are wrong, we know what we need to do:
  • Fix the whichNormalized function to give us the right path in the right context (and possible not use sh.which anymore). Might have to ask on the VSCode issue page, if there is a hidden system tools API to get the correct absolute path, depending on context?
• The way I see it, the only viable alternative solution is a "universal shell script converter", if you feel like taking on a challenge (or maybe there is a very reliable library out there - dunno?).

Either way, I'll gladly merge all well-tested and properly coded PRs, and start adding a contributor section to the main page to feature anyone who provides such a thing. :)

[CherryDT]

I think, the easier way to move forward here is to somehow get VSCode to give us the "correct bash path" given the context. I.e., in a local VSCode instance, find out what we need to do to give us the correct bash on the local system. Conversely, in a WSL instance (is that how it works?), get VSCode (or the system or whoever) to give us the WSL bash instead. What do you think?

It seems there is still a misunderstanding.

Summary: If you run dbux on Windows (not in a remote WSL session, there it works!) but with WSL installed on the machine, it will attempt to use C:\Windows\System32\bash.exe which is not just bash, in fact it's a "portal" into Linux and starts bash there instead. You can compare it to ssh user@somelinuxserver.com!

What happens if you just execute bash? The OS will look for a binary with that name in a folder that's in your PATH and use the first one it finds. In Windows, it will look for different file extensions as well (.exe being one of them, this is defined in PATHEXT), while Linux won't do that, it has to be literally the given filename.

Let's look at what happens in a fresh Windows install: There is simply no bash.exe anywhere in PATH, so it won't work. OK, now what happens if I install Git, which comes with a MINGW/MSYS bash for Windows? Well, it still doesn't work, because bash won't be in the PATH (C:\Program Files\Git\cmd is by default added to PATH so git works, but C:\Program Files\Git\bin isn't)! Once we add it (or manually configure the path in your setting - but that then can cause confusion with the defaults in a WSL session due to the other issue in regards to configs described earlier), it works.

The problem starts however once we install WSL on the machine. Doing so adds a few additional binaries into C:\Windows\system32, one of them being bash.exe, which is not actually bash but just a launcher that will run /bin/bash in the default WSL distro. This is probably a remnant from the days when WSL was called "Bash on Windows" and running bash was the primary advertised purpose of it. Anyway, running this system-provided bash.exe will suddenly "teleport you" into a different operating system. To a certain degree you can imagine it like opening an SSH session (with some extra bells and whistles such as that the file system is to a certain degree shared - Windows' drives are mounted in Linux as well - and therefore the current working directory can usually be preserved). Now, since this bash.exe (that's not really just bash but a "portal" into Linux) is in C:\Windows\system32 which is quite early in the PATH list, you will now get that thing when you run bash (which is what Microsoft intended).

On the other hand, in Linux (including inside a remote VSCode session in WSL), things work fine from the get-go because bash is a standard component in almost every Linux distribution out there. It's usually located in /bin, which is by default in the PATH, so no issue there.

In 2021, a third of developers stated that they use WSL in some way - even if they don't use it for everything, it's enough that it's installed - and given that this scenario happens by default and not as some strange configuration edge case, I think there should be a way to prevent it.

I will look into my idea of making it work independently of the shell used but another, simpler solution would be to just guide the user out of this trap - if your OS is Windows, check what shell you got (bash --version), if it's not what you need then explain what shell you expect exactly (a MINGW/MSYS-style bash on Windows) and ask for the path to it (maybe start by guessing it from Git - if you get the full path to git.exe from PATH and then go to ..\bin\bash.exe from there you should find it), and if your OS is Linux and you see a bash path in your extension settings that contains backslashes then assume it's from the setting in Windows and ignore it because simply bash will work in Linux anyway (this should solve the confusing thing about the handling of defaults in VSCode configurations from before).

[Domiii]

I see, so it is a system-wide setting. Also I was not aware of how popular WSL has gotten. Gotcha (or at least I hope, i got it - hah)!

• if your OS is Windows, check what shell you got (bash --version)

  • In theory, that can just be done in checkSystem.
  • This is also the thing we use for the "Check System Dependencies" command, as well as some other places. e.g. when clicking the Run Button.
  • But sadly, I think... the way it works now is that we are neglecting to check this before trying to do the install thing, which means, we cannot yet run it before install has completed. So in addition to fixing checkSystem, we also have to make sure, it runs in installDbuxDependencies, and before the actual installation begins.
  • But for that to work, we need to fix or remove the getProjectManager dependency (the dependency does not make too much sense, consider it legacy). Instead, we probably want to refactor and move all install-related code into a new DbuxInstallmanager which does not depend on projectManager.
  • Maybe it is actually not strictly necessary to do so, but that would be an "ideal" solution. Maybe it can be done with just changing a few lines already. Not sure yet.
  • If you want to try it, you are very welcome! If you get stuck, feel free to reach out. And if you feel its too much, we'll try to do it in the next few weeks or so!

[Domiii]

• now what happens if I install Git, which comes with a MINGW/MSYS bash for Windows? Well, it still doesn't work

  • It does for me. Not sure, if things changed, but all Git installers usually ask you whether you want to add UNIX tools to path. We even explain it in our docs here.

[CherryDT]

OK yes if you do that, it does, but it's not recommended because it overwrites other tools like find. Also, if you use the "modern way" Windows provides for a while, using Windows' built-in package manager WinGet, you won't get asked. (Note that even the Git download page shows the WinGet command line you can use.)

[Domiii]

Again, much appreciated that you bring all of this to my attention!

Considering the ever-growing mountain of problems with bash, maybe this can be a better sln:

1. Check the system.
   • On win: use cmd (Win11 still has cmd, right? 😅)
   • Everywhere else: use bash
   • → I think this might just work, since, for the commands we use, I believe, cmd and bash actually behave identically. Only PS loves to mess everything up.
2. We would allow overriding both: paths.cmd and paths.bash in config (i.e. "windows shell" and "non-windows shell"), just to be safe.

Two questions:

1. Do you think, this would be a solution that would work with WSL, too (not sure if I am overseeing anything)?
2. Also, just to verify: If you want to do stuff on the remote, you would run a separate instance of VSCode that actually resides on the linux side of things, meaning it itself does not even know that it is in WSL or windows, it thinks it is in Linux - is that correct?

[CherryDT]

Do you think, this would be a solution that would work with WSL, too (not sure if I am overseeing anything)?

It shouldn't interact with WSL in any way so that should work fine! (Yes only PS is the odd one because it's more of a programming language than a shell, really)

Sounds like a great solution to me.

Also, just to verify: If you want to do stuff on the remote, you would run a separate instance of VSCode that actually resides on the linux side of things, meaning it itself does not even know that it is in WSL or windows, it thinks it is in Linux - is that correct?

Parts of VSCode would run in WSL, yes - and those wouldn't know (unless they specifically check for it) that they are in WSL, it is simply Linux (Ubuntu in my case). Everything is transparent. The parts that don't run in WSL would be the UI and extensions that somehow declare they want that, as far as I know. But despite the UI running in Windows, your extension works and displays fine in the WSL remote window, and the file paths it passes to the node commands are actually Linux paths, so things seem to behave as expected. From what I understand (don't quote me on that) in VSCode different things run in different processes anyway, like the actual UI is in a different process than the extension host, and those processes communicate with one another, and in case of the remote features (either in WSL or Docker or a remote server via SSH) VSCode will run a separate "server" component on the remote side that will run its own extension host and such, but it will communicate with the main VSCode application running on my machine in Windows in such a way that the stuff running in Linux doesn't know it's actually talking to a VSCode instance in another system for things like rendering UI elements, things just work as they normally would.

[Domiii]

Ok, thanks for the explanation... We will put the new no-bash solution on our TODO list. It might come in the next few weeks :)

[Domiii]

@CherryDT Good news. After you helped us understand how wrong our assumptions were about bash on windows, I realized that there will definitely be a good amount of people who would be hampered by this issue. Also, in previous experiments, getting bash right was definitely an issue that slowed people down. That is why I decided today to just rip the bandage off and get this done.

In the process, we had to discover and tussle with this volta bug - cross-platform dev is such fun!

Either way - I finished the "use cmd on windows" solution today, will test more, and then deploy in the next few days!

[Domiii]

Its deployed. Working ok so far.