@testitem does not run - Githubissues

lmiq commented 1 year ago

Followup from: https://discourse.julialang.org/t/prerelease-of-new-testing-framework-and-test-run-ui-in-vs-code/86355/10?u=lmiq

The “play” button not doing anything in this package, which is only the template plus the tests: GitHub - lmiq/TestTests.jl

I’m on Linux Mint 21 x86 - with Julia 1.8.0.

There is no output at all, as the figure above shows, the spinning stuff just keeps spinning.

There is nothing particular about the Julia extension configuration (I think I didn't change anything).

To try to reproduce the problem, do:

git clone https://github.com/lmiq/TestTests.jl
cd TestTests.jl
code .

VSCode will open. Wait for the tests to appear at the Erlenmeyer tab, and click play.

carstenbauer commented 1 year ago

I see the same thing.

davidanthoff commented 1 year ago

Hm, so it all works on my Windows system with this repo... So we'll have to dig a bit :)

Here are some ideas:

Could you package the entire folder as it is locally on your hard drive and share? In particular, is there a Manifest.toml in the package root folder? I see in your status bar that the project TestTests.jl is active, how did you activate that?
Do you have anything configured for Julia: Executable Path? If yes, could you share it?
How is Julia installed, and in particular how did you put julia on the PATH on your system?

My best guess is that something goes haywire when the extension tries to start the test process already, i.e. maybe it doesn't even ever start, or something like that. At least in theory, that output panel you can open with that button should just show all and every output from the test process, so the fact that it doesn't show anything makes me think that we somehow fail to launch it already...

If either of you has the ability to run the entire Julia extension under the debugger, I could also give you some more pointers on things to check.

davidanthoff commented 1 year ago

Ah, another question: do you have a startup.jl?

davidanthoff commented 1 year ago

Ah, and I totally forgot: there is also an output channel for the test stuff that might show more. I doubt it, but you could check. Ctrl+Shift+U, and then there should be one called "Julia Testserver".

lmiq commented 1 year ago

Could you package the entire folder as it is locally on your hard drive and share? In particular, is there a Manifest.toml in the package root folder? I see in your status bar that the project TestTests.jl is active, how did you activate that?

I can reproduce the error cloning the repo again from scratch, so that repo, in that sense, is complete to reproduce the problem.

The activation of the environment there happened after VSCode asked me to activate it, but I suspected of that and tried without activating it as well, and the result is the same.

Do you have anything configured for Julia: Executable Path? If yes, could you share it?

No, only julia, and it was installed with juliaup.

leandro@pitico:~% which julia
/home/leandro/.juliaup/bin/julia

And it is in the path by the default juliaup addition to .bashrc.

Ah, another question: do you have a startup.jl?

I have, but I removed it now and nothing changed.

Ctrl+Shift+U, and then there should be one called "Julia Testserver".

Not there. But in the

Log (Window) this appears:

[2022-08-27 17:45:11.104] [renderer1] [error] spawn julia ENOENT: Error: spawn julia ENOENT
    at Process.ChildProcess._handle.onexit (node:internal/child_process:282:19)
    at onErrorNT (node:internal/child_process:477:16)
    at processTicksAndRejections (node:internal/process/task_queues:83:21)

And this is the only error message that appears upon clicking on the Play button of the test.

davidanthoff commented 1 year ago

Ah, that suggests that it is related to spawning Julia! So just to be double sure:

This whole output panel doesn't show up for you? Or is there just no content?

lmiq commented 1 year ago

The panel is there, but not the content:

lmiq commented 1 year ago

If either of you has the ability to run the entire Julia extension under the debugger,

If you need this, let me know, I won't be able to do it now, but during the week we can work on that.

davidanthoff commented 1 year ago

If you need this, let me know

Thanks, much appreciated!

I'm prepping a new build of the extension that would send a crash report with more details if it can't spawn the test server. My hope is that that would have enough information in it that we don't have to go down that route. But if that doesn't work, I might come back to your offer :) I'll ping you here as soon as this new extension build is on the insider channel of the extension.

davidanthoff commented 1 year ago

Ok, I've pushed an insider build out. @lmiq and @carstenbauer could you try to uninstall the regular extension and the install the Julia Insider one?

It should a) show a bit more output and errors and b) send crash reports that hopefully will help.

lmiq commented 1 year ago

Here it goes what I get. First, what appears in the interface:

Now the messages in the Output panel of each section:

Nothing at: Tasks, Extensions, Julia Notebook Kernels, Julia Test Server, Julia Language Server Trace

Julia Language Server

Juliaup configuration is locked by another process, waiting for it to unlock.
Activating project at `~/.vscode/extensions/julialang.language-julia-insider-1.7.7/scripts/environments/languageserver/v1.8`
[ Info: Starting the Julia Language Server
[ Info: Symbol server store is at '/home/leandro/.config/Code/User/globalStorage/julialang.language-julia-insider/symbolstorev5'.
[ Info: Starting LS at 1661687790
[ Info: All cache files downloaded.
[ Info: Received new data from Julia Symbol Server.

Log (Edit sessions):

[2022-08-28 08:54:57.944] [editsessions] [info] Skipping registering edit sessions actions as edit sessions are currently disabled. Set workbench.experimental.editSessions.enabled to enable edit sessions.

Log (Window):

[2022-08-28 08:54:59.955] [renderer1] [error] [Extension Host] (node:505641) DeprecationWarning: uuidv4() is deprecated. Use v4() from the uuid module instead.
(Use `code --trace-deprecation ...` to show where the warning was created)

Log (Main)

[2022-08-28 08:54:56.838] [main] [info] update#setState idle
[2022-08-28 08:54:58.128] [main] [info] Starting extension host with pid 505641 (fork() took 11 ms).
[2022-08-28 08:55:26.850] [main] [info] update#setState checking for updates
[2022-08-28 08:55:27.086] [main] [info] update#setState idle

Log (Extension Host)

[2022-08-28 08:54:58.817] [exthost] [info] Extension host with pid 505641 started
[2022-08-28 08:54:58.817] [exthost] [info] Skipping acquiring lock for /home/leandro/.config/Code/User/workspaceStorage/a84d09ffe692b1fbd58e9fb9cc983ebd.
[2022-08-28 08:54:58.949] [exthost] [info] ExtensionService#_doActivateExtension vscode.microsoft-authentication, startup: false, activationEvent: 'onAuthenticationRequest:microsoft'
[2022-08-28 08:54:58.968] [exthost] [info] ExtensionService#_doActivateExtension vscode.debug-auto-launch, startup: true, activationEvent: '*'
[2022-08-28 08:54:58.971] [exthost] [info] ExtensionService#_doActivateExtension vscode.git-base, startup: true, activationEvent: '*', root cause: vscode.git
[2022-08-28 08:54:58.975] [exthost] [info] ExtensionService#_doActivateExtension vscode.ipynb, startup: true, activationEvent: '*'
[2022-08-28 08:54:58.995] [exthost] [info] ExtensionService#_doActivateExtension vscodevim.vim, startup: true, activationEvent: '*'
[2022-08-28 08:54:59.059] [exthost] [info] ExtensionService#_doActivateExtension julialang.language-julia-insider, startup: true, activationEvent: 'workspaceContains:Project.toml'
[2022-08-28 08:54:59.940] [exthost] [info] ExtensionService#_doActivateExtension vscode.git, startup: true, activationEvent: '*'
[2022-08-28 08:55:00.065] [exthost] [info] ExtensionService#_doActivateExtension vscode.github, startup: true, activationEvent: '*'
[2022-08-28 08:55:00.185] [exthost] [info] ExtensionService#_doActivateExtension vscode.github-authentication, startup: false, activationEvent: 'onAuthenticationRequest:github'
[2022-08-28 08:55:00.380] [exthost] [info] Eager extensions activated
[2022-08-28 08:55:00.399] [exthost] [info] ExtensionService#_doActivateExtension vscode.emmet, startup: false, activationEvent: 'onStartupFinished'
[2022-08-28 08:55:00.406] [exthost] [info] ExtensionService#_doActivateExtension vscode.merge-conflict, startup: false, activationEvent: 'onStartupFinished'

Log (Shared)

[2022-08-28 08:54:59.081] [sharedprocess] [info] Starting worker process with pid 505675 (type: fileWatcher, window: 1).
[2022-08-28 08:54:59.407] [sharedprocess] [info] Deleted from disk julialang.language-julia /home/leandro/.vscode/extensions/julialang.language-julia-1.7.6

Log (Settings sync)

[2022-08-28 08:54:58.244] [settingssync] [info] Using settings sync service https://vscode-sync.trafficmanager.net/
[2022-08-28 08:54:58.244] [settingssync] [info] Auto Sync is disabled.

lmiq commented 1 year ago

When I cancel the test execution, this appears:

lmiq commented 1 year ago

I get this when I click "Play" on the test after starting the debugger (maybe just an environment issue?)

Then I installed the TestItemRunner package in the shared/global enviroment, and now I got this:


┌ Error: Some Julia code in the VS Code extension crashed
└ @ VSCodeDebugger ~/.vscode/extensions/julialang.language-julia-insider-1.7.7/scripts/error_handler.jl:15
ERROR: ArgumentError: lowering did not return a `:thunk` expression, got nothing
Stacktrace:
 [1] VSCodeDebugger.JuliaInterpreter.Frame(mod::Module, ex::Expr)
   @ VSCodeDebugger.JuliaInterpreter ~/.vscode/extensions/julialang.language-julia-insider-1.7.7/scripts/packages/JuliaInterpreter/src/types.jl:283
 [2] get_next_top_level_frame(state::VSCodeDebugger.DebugAdapter.DebuggerState)
   @ VSCodeDebugger.DebugAdapter ~/.vscode/extensions/julialang.language-julia-insider-1.7.7/scripts/packages/DebugAdapter/src/debugger_core.jl:60
 [3] our_debug_command(cmd::Symbol, state::VSCodeDebugger.DebugAdapter.DebuggerState)
   @ VSCodeDebugger.DebugAdapter ~/.vscode/extensions/julialang.language-julia-insider-1.7.7/scripts/packages/DebugAdapter/src/debugger_core.jl:83
 [4] startdebug(socket::Base.PipeEndpoint, error_handler::VSCodeDebugger.var"#3#4"{Tuple{String, String}})
   @ VSCodeDebugger.DebugAdapter ~/.vscode/extensions/julialang.language-julia-insider-1.7.7/scripts/packages/DebugAdapter/src/packagedef.jl:106
 [5] startdebugger()
   @ VSCodeDebugger ~/.vscode/extensions/julialang.language-julia-insider-1.7.7/scripts/packages/VSCodeDebugger/src/VSCodeDebugger.jl:43
 [6] top-level scope
   @ ~/.vscode/extensions/julialang.language-julia-insider-1.7.7/scripts/debugger/run_debugger.jl:10
 [7] include(mod::Module, _path::String)
   @ Base ./Base.jl:419
 [8] exec_options(opts::Base.JLOptions)
   @ Base ./client.jl:303
 [9] _start()
   @ Base ./client.jl:522

lmiq commented 1 year ago

Also got these errors after a restart of VSCode:

ERROR: LoadError: IOError: connect: no such file or directory (ENOENT)
Stacktrace:
 [1] wait_connected(x::Base.PipeEndpoint)
   @ Sockets ~/.julia/juliaup/julia-1.8.0+0.x64/share/julia/stdlib/v1.8/Sockets/src/Sockets.jl:529
 [2] connect
   @ ~/.julia/juliaup/julia-1.8.0+0.x64/share/julia/stdlib/v1.8/Sockets/src/Sockets.jl:564 [inlined]
 [3] connect
   @ ~/.julia/juliaup/julia-1.8.0+0.x64/share/julia/stdlib/v1.8/Sockets/src/PipeServer.jl:103 [inlined]
 [4] serve(args::String; is_dev::Bool, crashreporting_pipename::String)
   @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-insider-1.7.7/scripts/packages/VSCodeServer/src/VSCodeServer.jl:110
 [5] top-level scope
   @ ~/.vscode/extensions/julialang.language-julia-insider-1.7.7/scripts/terminalserver/terminalserver.jl:39
in expression starting at /home/leandro/.vscode/extensions/julialang.language-julia-insider-1.7.7/scripts/terminalserver/terminalserver.jl:14
julia> 
 *  History restored 

ERROR: LoadError: IOError: connect: no such file or directory (ENOENT)
Stacktrace:
 [1] wait_connected(x::Base.PipeEndpoint)
   @ Sockets ~/.julia/juliaup/julia-1.8.0+0.x64/share/julia/stdlib/v1.8/Sockets/src/Sockets.jl:529
 [2] connect
   @ ~/.julia/juliaup/julia-1.8.0+0.x64/share/julia/stdlib/v1.8/Sockets/src/Sockets.jl:564 [inlined]
 [3] connect
   @ ~/.julia/juliaup/julia-1.8.0+0.x64/share/julia/stdlib/v1.8/Sockets/src/PipeServer.jl:103 [inlined]
 [4] serve(args::String; is_dev::Bool, crashreporting_pipename::String)
   @ VSCodeServer ~/.vscode/extensions/julialang.language-julia-insider-1.7.7/scripts/packages/VSCodeServer/src/VSCodeServer.jl:110
 [5] top-level scope
   @ ~/.vscode/extensions/julialang.language-julia-insider-1.7.7/scripts/terminalserver/terminalserver.jl:39
in expression starting at /home/leandro/.vscode/extensions/julialang.language-julia-insider-1.7.7/scripts/terminalserver/terminalserver.jl:14
julia>

davidanthoff commented 1 year ago

Alright, this was super helpful, and I think I know what is going on and I also think I fixed. Turns out, the root cause is actually a bug entirely unrelated to testing that makes it that the extension will not detect juliaup properly on non Windows systems, and then also has problems spawning Julia when the launch command is just julia. I merged a fix that should just fix this for testing on master and just pushed out an insider build. @lmiq if you could retest your scenario with build 1.7.8 of the extension that would be great!

https://github.com/julia-vscode/julia-vscode/pull/3032 is a more thorough fix for the Juliaup integration bug. I'm waiting for @pfitzseb to review that before I merge it because it is going to affect a lot more scenarios than this preview test release.

Thanks for the fantastic bug report :)

lmiq commented 1 year ago

No, it does not run :-(.

I don't get those error messages, though. Just when trying to launch the debugger, I get this:

[2022-08-29 07:47:05.736] [renderer1] [error] The "path" argument must be of type string. Received undefined: TypeError: The "path" argument must be of type string. Received undefined
    at new NodeError (node:internal/errors:371:5)
    at validateString (node:internal/validators:119:11)
    at basename (node:path:1309:5)
    at f.<anonymous> (/home/leandro/.vscode/extensions/julialang.language-julia-insider-1.7.8/src/debugger/juliaDebug.ts:260:22)
    at Generator.next (<anonymous>)
    at a (/home/leandro/.vscode/extensions/julialang.language-julia-insider-1.7.8/src/debugger/debugProtocol.ts:209:90)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)

lmiq commented 1 year ago

Ah, on Julia Testserver output I got:

/bin/sh: 1: julia: not found

lmiq commented 1 year ago

Probably you already know that, but the tests work if I change the path on the extension settings to the full /home/leandro/.juliaup/bin/julia path.

davidanthoff commented 1 year ago

@lmiq sorry that I disappeared, but I'm back now and my goal is to fix this for good now.

One quick question: I assume starting a Julia REPL with Julia: Start REPL works just fine on your system, even if you don't configure an explicit path for Julia, right?

lmiq commented 1 year ago

, even if you don't configure an explicit path for Julia, right?

Yes, it always worked.

davidanthoff commented 1 year ago

@lmiq I pushed another insider build out, v1.7.9. It won't fix anything but has more diagnostics. Could you give that one a try? Make sure that you don't have anything configured as your Julia binary, i.e. let the extension auto-find everything. And then there should be a new output channel named just "Julia", and that should show (after the extension has started) some on how the extension found a Julia binary. Could you post that info here? Thanks, in particular for being so patient with this!

lmiq commented 1 year ago

I get:

and:

I removed everything from my .bashrc to avoid confusion.

Strangely, launching with julia works both in bash and sh, from a terminal.

lmiq commented 1 year ago

Now I removed the julia from the executable configuration, and left it blank, and it worked:

davidanthoff commented 1 year ago

@lmiq so the situation is that

if you leave the Julia: Executable Path setting empty it all works
if you have a value of julia in there it still does not work, right?

Did you ever have a situation where the setting was empty and things did not work? Or did you always have julia in there as a setting?

I removed everything from my .bashrc to avoid confusion.

What exactly did you remove there? Were there any Juliaup code changes in that file? If so, I don't think anything will work if you remove that...

lmiq commented 1 year ago

if you leave the Julia: Executable Path setting empty it all works

Yes.

if you have a value of julia in there it still does not work, right?

Yes, correct, and I get that /bin/sh: 1: julia: not found error.

Did you ever have a situation where the setting was empty and things did not work? Or did you always have julia in there as a setting?

No, I always had julia there (not that I remember changing that).

What exactly did you remove there? Were there any Juliaup code changes in that file? If so, I don't think anything will work if you remove that...

There was, but juliaup also modified the ~/.profile, thus the changes that it made to .bashrc were redundant. (I discovered that when I erased the .bashrc content and was surprised that julia was still in the path).*

(just to comment: I have been testing this in two different machines - both Linux Mint with similar specifications, but installed at very different moments in time, and the behavior is consistently the same in both).

*the content is, just for the records:

# >>> juliaup initialize >>>

# !! Contents within this block are managed by juliaup !!

case ":$PATH:" in *:/home/leandro/.juliaup/bin:*);; *)
    export PATH=/home/leandro/.juliaup/bin${PATH:+:${PATH}};;
esac

# <<< juliaup initialize <<<

Another curiosity (that surprised me given the /bin/sh error, is that I can launch Julia from sh normally - thanks to the modifications of juliaup to the ~/.profile file, I believe:

leandro@m3g:~%  sh
$ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.8.0 (2022-08-17)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia>

My first guess was that I would get a julia not found error here, but I don't.

davidanthoff commented 1 year ago

What does which julia show? and which juliaup?

lmiq commented 1 year ago

Sorry, I was lecturing:

leandro@m3g:~/Drive/Disciplinas/qf531% which julia
/home/leandro/.juliaup/bin/julia
leandro@m3g:~/Drive/Disciplinas/qf531% which juliaup
/home/leandro/.juliaup/bin/juliaup

davidanthoff commented 1 year ago

Alright, I just pushed a new build (1.7.10) to the insider channel. @lmiq could you try that one again? I think I did find a bug that would explain everything you have observed and hopefully have fixed that now. With build 1.7.10 everything should work, i.e. the situation with an empty exec path setting and with just julia in there.

lmiq commented 1 year ago

Yes, that solved the problem!

Thank you very much for all that, the test extension is absolutely great!

davidanthoff commented 1 year ago

Hurray! @lmiq thanks so much for helping me figure this out, fantastic bug reports and follow ups like yours here are just invaluable.

julia-vscode / TestItemRunner.jl

@testitem does not run #19