search

JuliaLang / Pkg.jl

Pkg - Package manager for the Julia programming language
https://pkgdocs.julialang.org
Other
613 stars 251 forks source link

Base.runtests(["Pkg"]) hangs #3774

Open vtjnash opened 5 months ago

vtjnash commented 5 months ago 
julia> Base.runtests(["Pkg"])
Running parallel tests with:
  getpid() = 5539
  nworkers() = 1
  nthreads() = 1
  Sys.total_memory() = 7.274 GiB
  Sys.free_memory() = 1.777 GiB

Test  (Worker) | Time (s) | GC (s) | GC % | Alloc (MB) | RSS (MB)
Pkg        (1) |        started at 2023-12-17T08:10:38.155
┌ Info: Pkg Server metadata:
│ HTTP/2 301 
│ server: Varnish
│ retry-after: 0
│ location: https://us-east.pkg.julialang.org/
[...]
    Updating `/tmp/jl_CXNkw2/environments/v1.11/Project.toml`
  [7876af07] + Example v0.5.4 `https://github.com/JuliaLang/Example.jl#master`
    Updating `/tmp/jl_CXNkw2/environments/v1.11/Manifest.toml`
  [7876af07] + Example v0.5.4 `https://github.com/JuliaLang/Example.jl#master`
    Updating `/tmp/jl_CXNkw2/environments/v1.11/Project.toml`
  [7876af07] - Example v0.5.4 `https://github.com/JuliaLang/Example.jl#master`
    Updating `/tmp/jl_CXNkw2/environments/v1.11/Manifest.toml`
  [7876af07] - Example v0.5.4 `https://github.com/JuliaLang/Example.jl#master`
     Cloning git-repo `https://github.com/DoesNotExist/DoesNotExist.jl`
Username for 'https://github.com':

Base.runtests(["sorting", "Pkg"]; ncores=2), on the other hand, works fine. Presumably because in the latter case Pkg tests are run in a separate worker with a stdin other than stdin, so Pkg can detect that prompting for credentials is fruitless.

Originally posted by @LilithHafner in https://github.com/JuliaLang/julia/issues/52562

IanButterworth commented 5 months ago

To fix this we can either:

vtjnash commented 5 months ago

GIT_TERMINAL_PROMPT=0? Or GIT_ASKPASS=true

IanButterworth commented 5 months ago

We already set the former. I can try the latter

vtjnash commented 5 months ago

Still hangs. After mucking around a bit with the process, I ended up being able to extract this trace:

     ---- Task 4 (0x7ffaa52a92d0)
          (sticky: 1, started: 1, state: 0, tid: 1)
jl_swap_fiber at /home/vtjnash/julia/src/task.c:1366
ctx_switch at /home/vtjnash/julia/src/task.c:571
ijl_switch at /home/vtjnash/julia/src/task.c:650
try_yieldto at ./task.jl:934
wait at ./task.jl:1008
#wait#720 at ./condition.jl:130
wait at ./condition.jl:125 [inlined]
wait_readnb at ./stream.jl:416
eof at ./stream.jl:106
iterate at ./io.jl:1380 [inlined]
_copyuntil at ./io.jl:1005
#copyuntil#493 at ./io.jl:994 [inlined]
copyuntil at ./io.jl:994 [inlined]
#copyline#517 at ./iobuffer.jl:587 [inlined]
copyline at ./iobuffer.jl:586 [inlined]
#readline#483 at ./io.jl:617
readline at ./io.jl:617 [inlined]
#prompt#1107 at ./util.jl:391
prompt at ./util.jl:388
unknown function (ip: 0x7ffa8e5b77d2)
_jl_invoke at /home/vtjnash/julia/src/gf.c:2941 [inlined]
ijl_apply_generic at /home/vtjnash/julia/src/gf.c:3118
#prompt#1108 at ./util.jl:399
prompt at ./util.jl:399 [inlined]
authenticate_userpass at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/LibGit2/src/callbacks.jl:217
credentials_callback at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/LibGit2/src/callbacks.jl:332
credentials_callback at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/LibGit2/src/callbacks.jl:355
unknown function (ip: 0x7ffa8e5b6e61)
_jl_invoke at /home/vtjnash/julia/src/gf.c:2941 [inlined]
ijl_apply_generic at /home/vtjnash/julia/src/gf.c:3118
jlcapi_cfunction_45231 at /home/vtjnash/julia/usr/share/julia/compiled/v1.11/Pkg/tUTdb_OHcd8.so (unknown line)
handle_auth at /home/vtjnash/julia/usr/bin/../lib/libgit2.so.1.7 (unknown line)
handle_response.constprop.5 at /home/vtjnash/julia/usr/bin/../lib/libgit2.so.1.7 (unknown line)
http_stream_read at /home/vtjnash/julia/usr/bin/../lib/libgit2.so.1.7 (unknown line)
git_smart__recv at /home/vtjnash/julia/usr/bin/../lib/libgit2.so.1.7 (unknown line)
git_smart__store_refs at /home/vtjnash/julia/usr/bin/../lib/libgit2.so.1.7 (unknown line)
git_smart__connect at /home/vtjnash/julia/usr/bin/../lib/libgit2.so.1.7 (unknown line)
git_remote_connect_ext at /home/vtjnash/julia/usr/bin/../lib/libgit2.so.1.7 (unknown line)
git__clone at /home/vtjnash/julia/usr/bin/../lib/libgit2.so.1.7 (unknown line)
macro expansion at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/LibGit2/src/error.jl:113 [inlined]
clone at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/LibGit2/src/repository.jl:459
#clone#123 at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/LibGit2/src/LibGit2.jl:585
clone at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/LibGit2/src/LibGit2.jl:558 [inlined]
#clone#3 at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/src/GitTools.jl:121
clone at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/src/GitTools.jl:88
unknown function (ip: 0x7ffa8e5bafdd)
_jl_invoke at /home/vtjnash/julia/src/gf.c:2941 [inlined]
ijl_apply_generic at /home/vtjnash/julia/src/gf.c:3118
jl_apply at /home/vtjnash/julia/src/julia.h:2165 [inlined]
do_apply at /home/vtjnash/julia/src/builtins.c:831
#103#104 at ./error.jl:298
#103 at ./error.jl:293 [inlined]
#50 at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/src/Registry/Registry.jl:238
#mktempdir#28 at ./file.jl:819
unknown function (ip: 0x7ffa8e5ba06d)
_jl_invoke at /home/vtjnash/julia/src/gf.c:2941 [inlined]
ijl_apply_generic at /home/vtjnash/julia/src/gf.c:3118
mktempdir at ./file.jl:815
mktempdir at ./file.jl:815 [inlined]
#48 at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/src/Registry/Registry.jl:202
#mkpidlock#7 at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/FileWatching/src/pidfile.jl:95
mkpidlock at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/FileWatching/src/pidfile.jl:92 [inlined]
#mkpidlock#6 at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/FileWatching/src/pidfile.jl:90 [inlined]
mkpidlock at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/FileWatching/src/pidfile.jl:90 [inlined]
download_registries at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/src/Registry/Registry.jl:168
#add#34 at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/src/Registry/Registry.jl:47
unknown function (ip: 0x7ffa8e5b788e)
_jl_invoke at /home/vtjnash/julia/src/gf.c:2941 [inlined]
ijl_apply_generic at /home/vtjnash/julia/src/gf.c:3118
add at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/src/Registry/Registry.jl:43
#add#31 at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/src/Registry/Registry.jl:40 [inlined]
add at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/src/Registry/Registry.jl:40 [inlined]
macro expansion at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/test/new.jl:2754 [inlined]
macro expansion at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Test/src/Test.jl:773 [inlined]
macro expansion at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/test/new.jl:2754 [inlined]
macro expansion at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Test/src/Test.jl:1700 [inlined]
#708 at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/test/new.jl:2752
#10 at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/test/utils.jl:82
withenv at ./env.jl:265
#isolate#9 at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/test/utils.jl:73
isolate at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/test/utils.jl:51 [inlined]
macro expansion at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/test/new.jl:2735 [inlined]
macro expansion at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Test/src/Test.jl:1700 [inlined]
#707 at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/test/new.jl:2735
withenv at ./env.jl:265
_jl_invoke at /home/vtjnash/julia/src/gf.c:2941 [inlined]
ijl_apply_generic at /home/vtjnash/julia/src/gf.c:3118
jl_apply at /home/vtjnash/julia/src/julia.h:2165 [inlined]
do_call at /home/vtjnash/julia/src/interpreter.c:126
eval_value at /home/vtjnash/julia/src/interpreter.c:223
eval_stmt_value at /home/vtjnash/julia/src/interpreter.c:174 [inlined]
eval_body at /home/vtjnash/julia/src/interpreter.c:675
jl_interpret_toplevel_thunk at /home/vtjnash/julia/src/interpreter.c:815
top-level scope at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/test/new.jl:2733
jl_toplevel_eval_flex at /home/vtjnash/julia/src/toplevel.c:943
jl_eval_module_expr at /home/vtjnash/julia/src/toplevel.c:215 [inlined]
jl_toplevel_eval_flex at /home/vtjnash/julia/src/toplevel.c:743
jl_toplevel_eval_flex at /home/vtjnash/julia/src/toplevel.c:886
ijl_toplevel_eval_in at /home/vtjnash/julia/src/toplevel.c:994
eval at ./boot.jl:428 [inlined]
include_string at ./loading.jl:2260
_jl_invoke at /home/vtjnash/julia/src/gf.c:2941 [inlined]
ijl_apply_generic at /home/vtjnash/julia/src/gf.c:3118
_include at ./loading.jl:2320
include at ./Base.jl:556
jfptr_include_43348 at /home/vtjnash/julia/usr/lib/julia/sys.so (unknown line)
_jl_invoke at /home/vtjnash/julia/src/gf.c:2941 [inlined]
ijl_apply_generic at /home/vtjnash/julia/src/gf.c:3118
jl_apply at /home/vtjnash/julia/src/julia.h:2165 [inlined]
jl_f__call_latest at /home/vtjnash/julia/src/builtins.c:875
include at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/test/runtests.jl:10 [inlined]
macro expansion at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/test/runtests.jl:83 [inlined]
macro expansion at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Test/src/Test.jl:1789 [inlined]
macro expansion at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/test/runtests.jl:63 [inlined]
macro expansion at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Test/src/Test.jl:1700 [inlined]
#1 at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/test/runtests.jl:62
with_logstate at ./logging.jl:515
unknown function (ip: 0x7ffa99b57a53)
_jl_invoke at /home/vtjnash/julia/src/gf.c:2941 [inlined]
ijl_apply_generic at /home/vtjnash/julia/src/gf.c:3118
with_logger at ./logging.jl:625
unknown function (ip: 0x7ffa99b4bce6)
_jl_invoke at /home/vtjnash/julia/src/gf.c:2941 [inlined]
ijl_apply_generic at /home/vtjnash/julia/src/gf.c:3118
jl_apply at /home/vtjnash/julia/src/julia.h:2165 [inlined]
do_call at /home/vtjnash/julia/src/interpreter.c:126
eval_value at /home/vtjnash/julia/src/interpreter.c:223
eval_stmt_value at /home/vtjnash/julia/src/interpreter.c:174 [inlined]
eval_body at /home/vtjnash/julia/src/interpreter.c:675
jl_interpret_toplevel_thunk at /home/vtjnash/julia/src/interpreter.c:815
top-level scope at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Pkg/test/runtests.jl:37
jl_toplevel_eval_flex at /home/vtjnash/julia/src/toplevel.c:943
jl_eval_module_expr at /home/vtjnash/julia/src/toplevel.c:215 [inlined]
jl_toplevel_eval_flex at /home/vtjnash/julia/src/toplevel.c:743
jl_eval_module_expr at /home/vtjnash/julia/src/toplevel.c:215 [inlined]
jl_toplevel_eval_flex at /home/vtjnash/julia/src/toplevel.c:743
jl_toplevel_eval_flex at /home/vtjnash/julia/src/toplevel.c:886
ijl_toplevel_eval_in at /home/vtjnash/julia/src/toplevel.c:994
eval at ./boot.jl:428 [inlined]
include_string at ./loading.jl:2260
_jl_invoke at /home/vtjnash/julia/src/gf.c:2941 [inlined]
ijl_apply_generic at /home/vtjnash/julia/src/gf.c:3118
_include at ./loading.jl:2320
include at ./Base.jl:556 [inlined]
macro expansion at /home/vtjnash/julia/test/testdefs.jl:33 [inlined]
macro expansion at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Test/src/Test.jl:1700 [inlined]
macro expansion at /home/vtjnash/julia/test/testdefs.jl:26 [inlined]
macro expansion at ./timing.jl:578 [inlined]
#runtests#53 at /home/vtjnash/julia/test/testdefs.jl:24
runtests at /home/vtjnash/julia/test/testdefs.jl:5 [inlined]
runtests at /home/vtjnash/julia/test/testdefs.jl:5
unknown function (ip: 0x7ffa99b4166d)
_jl_invoke at /home/vtjnash/julia/src/gf.c:2941 [inlined]
ijl_apply_generic at /home/vtjnash/julia/src/gf.c:3118
jl_apply at /home/vtjnash/julia/src/julia.h:2165 [inlined]
jl_f__call_latest at /home/vtjnash/julia/src/builtins.c:875
#invokelatest#2 at ./essentials.jl:1027 [inlined]
invokelatest at ./essentials.jl:1022 [inlined]
#153 at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Distributed/src/remotecall.jl:425 [inlined]
run_work_thunk at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Distributed/src/process_messages.jl:70
unknown function (ip: 0x7ffa99b3159b)
_jl_invoke at /home/vtjnash/julia/src/gf.c:2941 [inlined]
ijl_apply_generic at /home/vtjnash/julia/src/gf.c:3118
#remotecall_fetch#158 at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Distributed/src/remotecall.jl:450
_jl_invoke at /home/vtjnash/julia/src/gf.c:2941 [inlined]
ijl_apply_generic at /home/vtjnash/julia/src/gf.c:3118
jl_apply at /home/vtjnash/julia/src/julia.h:2165 [inlined]
do_apply at /home/vtjnash/julia/src/builtins.c:831
remotecall_fetch at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Distributed/src/remotecall.jl:449
_jl_invoke at /home/vtjnash/julia/src/gf.c:2941 [inlined]
ijl_apply_generic at /home/vtjnash/julia/src/gf.c:3118
jl_apply at /home/vtjnash/julia/src/julia.h:2165 [inlined]
do_apply at /home/vtjnash/julia/src/builtins.c:831
#remotecall_fetch#162 at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Distributed/src/remotecall.jl:492
_jl_invoke at /home/vtjnash/julia/src/gf.c:2941 [inlined]
ijl_apply_generic at /home/vtjnash/julia/src/gf.c:3118
jl_apply at /home/vtjnash/julia/src/julia.h:2165 [inlined]
do_apply at /home/vtjnash/julia/src/builtins.c:831
remotecall_fetch at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/Distributed/src/remotecall.jl:492
_jl_invoke at /home/vtjnash/julia/src/gf.c:2941 [inlined]
ijl_apply_generic at /home/vtjnash/julia/src/gf.c:3118
#39 at /home/vtjnash/julia/test/runtests.jl:264
unknown function (ip: 0x7ffa99b30bdf)
_jl_invoke at /home/vtjnash/julia/src/gf.c:2941 [inlined]
ijl_apply_generic at /home/vtjnash/julia/src/gf.c:3118
jl_apply at /home/vtjnash/julia/src/julia.h:2165 [inlined]
start_task at /home/vtjnash/julia/src/task.c:1258
     ---- End task 4
IanButterworth commented 5 months ago

I'm not sure why we needed to move this issue discussion over, but in the other issue you said

According to gdb, the problem is in Pkg not in git.

but in the trace it shows

_copyuntil at ./io.jl:1005
#copyuntil#493 at ./io.jl:994 [inlined]
copyuntil at ./io.jl:994 [inlined]
#copyline#517 at ./iobuffer.jl:587 [inlined]
copyline at ./iobuffer.jl:586 [inlined]
#readline#483 at ./io.jl:617
readline at ./io.jl:617 [inlined]
#prompt#1107 at ./util.jl:391
prompt at ./util.jl:388
unknown function (ip: 0x7ffa8e5b77d2)
_jl_invoke at /home/vtjnash/julia/src/gf.c:2941 [inlined]
ijl_apply_generic at /home/vtjnash/julia/src/gf.c:3118
#prompt#1108 at ./util.jl:399
prompt at ./util.jl:399 [inlined]
authenticate_userpass at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/LibGit2/src/callbacks.jl:217
credentials_callback at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/LibGit2/src/callbacks.jl:332
credentials_callback at /home/vtjnash/julia/usr/share/julia/stdlib/v1.11/LibGit2/src/callbacks.jl:355

So it's the credentials prompt, no?