search

JuliaLang / julia

The Julia Programming Language
https://julialang.org/
MIT License
45.76k stars 5.49k forks source link

The "shell mode" in the REPL is not fully functional on Windows #23597

Open PetrKryslUCSD opened 7 years ago

PetrKryslUCSD commented 7 years ago

I know that this is not the fault of Julia, but the Windows shell does not play nicely with Julia in the "shell mode". For instance, typing ; dir will result in an error because this is not a command executed by program. This applies to most of the commands available in the Windows shell. So far I found only tree and help to work...

It would probably prevent some useless experimentation and disappointments for new users if this fact was clearly mentioned in the documentation: the "shell mode" does not work in Windows.

yuyichao commented 7 years ago

All commands (programs) should work. If you don't have any that's what "working" should be like so we would definitely not call it "not work".

PetrKryslUCSD commented 7 years ago

Sorry: what???c

TotalVerb commented 7 years ago

As mentioned, dir is not a program on Windows, which is why you cannot use it in shell mode. You can use instead cmd /e dir.

TotalVerb commented 7 years ago

I don't think it's correct to say the shell mode is not fully functional on Windows, but it is perhaps worth mentioning that on all operating systems, the shell is not necessarily bash or cmd and as such does not necessarily have support for bash or cmd builtins which are not actual programs. (For instance, on Linux I believe it uses /bin/sh, which is dash on Debian systems and dash supports a proper subset of bash builtins.)

yuyichao commented 7 years ago

Also note that the shell mode is not a native shell and is not meant to be so you shouldn't expect all shell (cmd) features to be available. I have no idea why dir is implemented as a builtin rather than a program but AFAIK it is so you won't be able to run any of them by design.

It was also decided against calling cmd automatically on windows since IIUC it has really weird escape/quoting behavior (again, I'm not exactly sure what those are but others might be able to give you examples). Powershell was also an option but IIUC availability was not as clear.

Still I agree the expectation should be documented and hence the doc label. It just shouldn't be called "not working".

yuyichao commented 7 years ago

Ref link discourse thread and I should clarify that my first comment assumed prior knowledge from that thread https://discourse.julialang.org/t/dir-does-not-work-in-the-console/5732/4?u=yuyichao (i.e. dir is not a program).

PetrKryslUCSD commented 7 years ago

@yuyichao is right, the code IS actually working as designed. Nevertheless, it will be a disappointment to those who expect the shell mode to be as usable on Windows as it is on Linux. Hence it may be better to be upfront about it in the documentation.

stevengj commented 7 years ago

See #21294 — it would be nicer to use cmd by default on Windows.

devel-chm commented 6 years ago

I'm new to Julia using win64 version 6.2 and just discovered that the Julia REPL "shell" mode doesn't work on windows, apparently by design. I have a few thoughts on this from my own experiences:

PetrKryslUCSD commented 6 years ago

The only solution on Windows so far is to run julia from within git bash. Then all the shell commands work properly. Setting JULIA_SHELL never worked on Windows AFAICT.

stevengj commented 6 years ago

It doesn't work because #23703 or similar was never merged.

devel-chm commented 6 years ago

Thanks for the tip. It appears to work from within the cygwin bash as well. I still think better documentation and some work to avoid confusion and unnecessary differences between the underlying hardware a software would be a good investment.

stevengj commented 6 years ago

I think it would be good to have an actual shell mode in Windows, rather than a "shell" mode that is not running any real shell.

epogrebnyak commented 6 years ago

@stevengj - I totally side with this expectation, if it called a shell it better be a shell.

On my side tree, help and cd are working, while echo and dir do not:

shell> echo 1
ERROR: IOError: could not spawn `echo 1`: no such file or directory (ENOENT)
Stacktrace:
 [1] _jl_spawn(::String, ::Array{String,1}, ::Cmd, ::Tuple{RawFD,RawFD,RawFD}) a
t .\process.jl:367
 [2] (::getfield(Base, Symbol("##495#496")){Cmd})(::Tuple{RawFD,RawFD,RawFD}) at
 .\process.jl:509
 [3] setup_stdio(::getfield(Base, Symbol("##495#496")){Cmd}, ::Tuple{RawFD,RawFD
,RawFD}) at .\process.jl:490
 [4] #_spawn#494(::Nothing, ::Function, ::Cmd, ::Tuple{RawFD,RawFD,RawFD}) at .\
process.jl:508
 [5] _spawn at .\process.jl:504 [inlined]
 [6] #run#505(::Bool, ::Function, ::Cmd) at .\process.jl:652
 [7] run at .\process.jl:651 [inlined]
 [8] repl_cmd(::Cmd, ::REPL.Terminals.TTYTerminal) at .\client.jl:77
 [9] top-level scope at none:0
epogrebnyak commented 6 years ago

Also cmd /e dir does not run dir, just starts a cmd session. I type exit to get back to julia.

vtjnash commented 6 years ago

cmd /e isn't a defined flag, so cmd.exe just ignores the arguments. You probably meant cmd /c dir (which does work as expected)

vtjnash commented 6 years ago

To act like a posix shell, there's a certain list of command strings that we need to intercept and not try to execute or pass to the real shell. Of these, we currently only define handling for cd, as the most useful. Enumeration of the others can be found via man bash:

bash defines the following built-in commands: :, ., [, alias, bg, bind, break,
       builtin, case, cd, command, compgen, complete, continue, declare, dirs,
       disown, echo, enable, eval, exec, exit, export, fc, fg, getopts, hash, help,
       history, if, jobs, kill, let, local, logout, popd, printf, pushd, pwd, read,
       readonly, return, set, shift,  shopt,  source,  suspend, test, times, trap,
       type, typeset, ulimit, umask, unalias, unset, until, wait, while.

There's a related list for cmd:

ASSOC, BREAK, CALL, CD/CHDIR, CLS, COLOR, COPY, DATE, DEL, DIR, DPATH,
ECHO, ENDLOCAL, ERASE, EXIT, FOR, FTYPE, GOTO, IF, KEYS, MD/MKDIR,
MKLINK (vista and above), MOVE, PATH, PAUSE, POPD, PROMPT, PUSHD, REM,
REN/RENAME, RD/RMDIR, SET, SETLOCAL, SHIFT, START, TIME, TITLE, TYPE,
VER, VERIFY, VOL
PetrKryslUCSD commented 6 years ago

All of this weirdness on Windows could be addressed by bundling git bash with Julia to handle the shell mode.

RossBoylan commented 5 years ago

If the current behavior on Windows is "working as intended" then the intentions were not well thought out, because they are out of sync with expectations of what a shell is.

People may differ in whether they expect a Windows command prompt, a bash-like shell, or powershell--it would be nice to have a choice--but I don't think anyone reading it's a shell would think it's none of the above.

venuur commented 4 years ago

I stumbled across this problem for the (I think) essentially same problem with run and back tick commands. Documentation seems to assume a Posix environment. It seems like it would be helpful to add a note that typical Windows parallel commands like dir will fail. I'll see if I can figure out how to make a pull request to add the note.

aminya commented 4 years ago

Is there a standard way to run a command that works both on Windows and Linux? This works on Linux, but not on Windows and I need pwsh /c or cmd /c for that

 run(`dir`)

I'll get:

ERROR: IOError: could not spawn `dir`: no such file or directory (ENOENT)

I have to explicitly use a shell command like cmd or pwsh

Can't we have a default shell command on windows? Like using pwsh /c or cmd /c by default if it fails?

function run_windows(command, shell::Union{String, Cmd} = `cmd`)
    try 
        run(cmd)
    catch e
         run(`$shell /c $command`)
    end
end
julia> run_windows(`dir`)
 Volume in drive C is Windows
...

julia> run_windows(`ls`, "pwsh")
    Directory: C:\Users\yahyaaba
PetrKryslUCSD commented 4 years ago

All of these problems could be solved by running Julia in a git bash. For instance, if you run vscode, atom, or sublime_text from a git bash, starting Julia in the editor will give you immediately access to a sound and sane shell (where run(`ls .`) just works).

aminya commented 4 years ago

All of these problems could be solved by running Julia in a git bash. For instance, if you run vscode, atom, or sublime_text from a git bash, starting Julia in the editor will give you immediately access to a sound and sane shell (where run(`ls .`) just works).

This is just similar to run(`bash -c $command`), which isn't a solution for all Windows versions. Not all have bash installed. The reason it works without writing bash -c when you open julia in bash is because the code written in Julia for run assumes bash shell commands

PetrKryslUCSD commented 4 years ago

I don't think the solution is to accommodate Windows weirdness. I think it is to replace it.

aminya commented 4 years ago

I don't think the solution is to accommodate Windows weirdness. I think it is to replace it.

I don't call this weirdness.

Microsoft Windows isn't a UNIX operating system, and its main terminal isn't POSIX compliant. Yes, there are ways to get Unix like functionality in Windows:

But this doesn't mean the main Windows terminal is weird, it is just another operating system (that historically existed way before Linux).

PetrKryslUCSD commented 4 years ago

We don't have to call it weirdness. I have no problem with that: but, the fact is Julia needs to be operated differently when on Windows rather than when running under Linux or MacOS. That's what I referred to. By running Julia as indicated above in a git bash, a uniform experience can be provided. So rather than have Julia compensate for the Windows behaviors, it could be run without change in a sane shell. Much less work, better results.

devel-chm commented 4 years ago

I'm enjoying julia 1.4.0 and have come back to work with the shell mode in the julia REPL. I believe that my confusion/dissatisfaction with the shell mode escape is that it isn't a "true unix-style" shell escape where a subshell is opened up. Exiting that subshell returns you to the parent process.

By running julia from a cygwin shell (in an xterm or from mintty) I get the current PATH so the basic commands work. However, the command environment is not another shell with all the aliases and functions that I expected. What seems to be equivalent to my expectations is to run the desired shell by hand and then in that context everything "just works". Exiting the shell resumes julia.

The functionality of the shell mode is more like an enhanced system() to run commands and so I think it would be better named as command mode rather than shell mode---although you can run a shell through it. If that were clearer in the explanation/docs that would have helped reduce my previous confusion.

Right now I'm trying to determine how difficult it would be to implement a true shell mode for the REPL so I can have my cake and eat it too.

bogdankjastrzebski commented 4 years ago

A short, very dirty solution is to add the Cygwin bin to your path. You'll get ls, cd etc. in cmd and hence in Julia. This is especially useful in Juno.

Ceterum censeo Windows esse delendam...

mas-co commented 2 years ago

I see comments claiming that it is not correct to say that the shell is not fully functional on Windows, yet there is a disparity and a lack of functionality. Just because Windows doesn't have an executable for every command, that, in itself, shouldn't break the shell. But it does. For example, Windows does not have a "pwd" command---it is aliased to the command "get-location". Yet "get-location" does not work in the Julia shell under windows. Do aliases work in the Julia shell as expected on the Unix and Mac platforms? If so, why not with Windows?

Personally, I have given up on Julia working with Windows' shell. I find it quite difficult to have to prefix everything with "CMD /e someCmd" or something similar every time I want to run a command. Eventually, I installed msys2, put that in the path, and Julia shell works well enough in most instances with the bash shell.

Having to do this makes it crystal clear to me that Julia shell is NOT fully functional on Windows.

CobaltMongoose commented 1 year ago

Just ran into this myself. The odd behavior on windows is unexpected. I could implement one of the above workarounds but personally I'm just going to treat it like a broken feature and not use it too much.

I think it would be great if a fix could close the user expectation gap for this feature.

xgdgsc commented 8 months ago

Would shipping together with shells like https://github.com/nushell/nushell be more practical? It could be the default on Windows and choosable on first run on Linux/macOS.

stevengj commented 8 months ago

@xgdgsc, yes, this has been proposed, e.g. in #23703 it was suggested to use busybox, but it never got a consensus.