CI: Intermittent failure in LinearAlgebra/qr

[Keno]

https://build.julialang.org/#/builders/42/builds/7682/steps/5/logs/stdio

      From worker 3:    
      From worker 3:    Please submit a bug report with steps to reproduce this fault, and any error messages that follow (in their entirety). Thanks.
      From worker 3:    Exception: EXCEPTION_ACCESS_VIOLATION at 0x6d3a994f -- gc_try_setmark at /cygdrive/c/buildbot/worker/package_win32/build/src\gc.c:1777 [inlined]
      From worker 3:    gc_mark_scan_obj8 at /cygdrive/c/buildbot/worker/package_win32/build/src\gc.c:2019 [inlined]
      From worker 3:    gc_mark_loop at /cygdrive/c/buildbot/worker/package_win32/build/src\gc.c:2315
      From worker 3:    in expression starting at C:\buildbot\worker-tabularasa\tester_win32\build\share\julia\stdlib\v1.9\LinearAlgebra\test\qr.jl:377
      From worker 3:    gc_try_setmark at /cygdrive/c/buildbot/worker/package_win32/build/src\gc.c:1777 [inlined]
      From worker 3:    gc_mark_scan_obj8 at /cygdrive/c/buildbot/worker/package_win32/build/src\gc.c:2019 [inlined]
      From worker 3:    gc_mark_loop at /cygdrive/c/buildbot/worker/package_win32/build/src\gc.c:2315
      From worker 3:    _jl_gc_collect at /cygdrive/c/buildbot/worker/package_win32/build/src\gc.c:3114
      From worker 3:    ijl_gc_collect at /cygdrive/c/buildbot/worker/package_win32/build/src\gc.c:3376
      From worker 3:    maybe_collect at /cygdrive/c/buildbot/worker/package_win32/build/src\gc.c:916 [inlined]
      From worker 3:    jl_gc_pool_alloc_inner at /cygdrive/c/buildbot/worker/package_win32/build/src\gc.c:1260 [inlined]
      From worker 3:    jl_gc_pool_alloc_noinline at /cygdrive/c/buildbot/worker/package_win32/build/src\gc.c:1319 [inlined]
      From worker 3:    jl_gc_alloc_ at /cygdrive/c/buildbot/worker/package_win32/build/src\julia_internal.h:416 [inlined]
      From worker 3:    jl_gc_alloc at /cygdrive/c/buildbot/worker/package_win32/build/src\gc.c:3421
      From worker 3:    _new_array_ at /cygdrive/c/buildbot/worker/package_win32/build/src\array.c:134 [inlined]
      From worker 3:    _new_array at /cygdrive/c/buildbot/worker/package_win32/build/src\array.c:198 [inlined]
      From worker 3:    ijl_alloc_array_2d at /cygdrive/c/buildbot/worker/package_win32/build/src\array.c:443
      From worker 3:    Array at .\boot.jl:477 [inlined]
      From worker 3:    Array at .\boot.jl:485 [inlined]
      From worker 3:    similar at .\array.jl:374 [inlined]
      From worker 3:    similar at .\abstractarray.jl:809 [inlined]
      From worker 3:    _unsafe_getindex at .\multidimensional.jl:873
      From worker 3:    _getindex at .\multidimensional.jl:861 [inlined]
      From worker 3:    getindex at .\abstractarray.jl:1267 [inlined]
      From worker 3:    ldiv! at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\LinearAlgebra\src\qr.jl:913
      From worker 3:    ldiv! at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\LinearAlgebra\src\qr.jl:921 [inlined]
      From worker 3:    ldiv! at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\LinearAlgebra\src\qr.jl:919
      From worker 3:    \ at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\LinearAlgebra\src\LinearAlgebra.jl:495
      From worker 3:    \ at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\LinearAlgebra\src\generic.jl:1112
      From worker 3:    macro expansion at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\Test\src\Test.jl:477 [inlined]
      From worker 3:    macro expansion at C:\buildbot\worker-tabularasa\tester_win32\build\share\julia\stdlib\v1.9\LinearAlgebra\test\qr.jl:405 [inlined]
      From worker 3:    macro expansion at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\Test\src\Test.jl:1492 [inlined]
      From worker 3:    macro expansion at C:\buildbot\worker-tabularasa\tester_win32\build\share\julia\stdlib\v1.9\LinearAlgebra\test\qr.jl:403 [inlined]
      From worker 3:    macro expansion at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\Test\src\Test.jl:1580 [inlined]
      From worker 3:    macro expansion at C:\buildbot\worker-tabularasa\tester_win32\build\share\julia\stdlib\v1.9\LinearAlgebra\test\qr.jl:382 [inlined]
      From worker 3:    macro expansion at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\Test\src\Test.jl:1580 [inlined]
      From worker 3:    macro expansion at C:\buildbot\worker-tabularasa\tester_win32\build\share\julia\stdlib\v1.9\LinearAlgebra\test\qr.jl:381 [inlined]
      From worker 3:    macro expansion at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\Test\src\Test.jl:1492 [inlined]
      From worker 3:    top-level scope at C:\buildbot\worker-tabularasa\tester_win32\build\share\julia\stdlib\v1.9\LinearAlgebra\test\qr.jl:378
      From worker 3:    jl_fptr_args at /cygdrive/c/buildbot/worker/package_win32/build/src\gf.c:2154
      From worker 3:    _jl_invoke at /cygdrive/c/buildbot/worker/package_win32/build/src\gf.c:2393 [inlined]
      From worker 3:    ijl_invoke at /cygdrive/c/buildbot/worker/package_win32/build/src\gf.c:2400
      From worker 3:    jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win32/build/src\toplevel.c:903
      From worker 3:    jl_eval_module_expr at /cygdrive/c/buildbot/worker/package_win32/build/src\toplevel.c:203 [inlined]
      From worker 3:    jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win32/build/src\toplevel.c:715
      From worker 3:    jl_toplevel_eval_flex at /cygdrive/c/buildbot/worker/package_win32/build/src\toplevel.c:856
      From worker 3:    ijl_toplevel_eval at /cygdrive/c/buildbot/worker/package_win32/build/src\toplevel.c:921 [inlined]
      From worker 3:    ijl_toplevel_eval_in at /cygdrive/c/buildbot/worker/package_win32/build/src\toplevel.c:971
      From worker 3:    eval at .\boot.jl:370 [inlined]
      From worker 3:    include_string at .\loading.jl:1491
      From worker 3:    jl_fptr_args at /cygdrive/c/buildbot/worker/package_win32/build/src\gf.c:2154
      From worker 3:    _jl_invoke at /cygdrive/c/buildbot/worker/package_win32/build/src\gf.c:2374 [inlined]
      From worker 3:    ijl_apply_generic at /cygdrive/c/buildbot/worker/package_win32/build/src\gf.c:2575
      From worker 3:    _include at .\loading.jl:1551
      From worker 3:    include at .\Base.jl:424 [inlined]
      From worker 3:    macro expansion at C:\buildbot\worker-tabularasa\tester_win32\build\share\julia\test\testdefs.jl:24 [inlined]
      From worker 3:    macro expansion at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\Test\src\Test.jl:1492 [inlined]
      From worker 3:    macro expansion at C:\buildbot\worker-tabularasa\tester_win32\build\share\julia\test\testdefs.jl:23 [inlined]
      From worker 3:    macro expansion at .\timing.jl:475 [inlined]
      From worker 3:    #runtests#3 at C:\buildbot\worker-tabularasa\tester_win32\build\share\julia\test\testdefs.jl:21
      From worker 3:    runtests##kw at C:\buildbot\worker-tabularasa\tester_win32\build\share\julia\test\testdefs.jl:5 [inlined]
      From worker 3:    runtests##kw at C:\buildbot\worker-tabularasa\tester_win32\build\share\julia\test\testdefs.jl:5
      From worker 3:    unknown function (ip: 2711653b)
      From worker 3:    _jl_invoke at /cygdrive/c/buildbot/worker/package_win32/build/src\gf.c:2393 [inlined]
      From worker 3:    ijl_apply_generic at /cygdrive/c/buildbot/worker/package_win32/build/src\gf.c:2575
      From worker 3:    jl_apply at /cygdrive/c/buildbot/worker/package_win32/build/src\julia.h:1852 [inlined]
      From worker 3:    jl_f__call_latest at /cygdrive/c/buildbot/worker/package_win32/build/src\builtins.c:774
      From worker 3:    jl_fptr_args at /cygdrive/c/buildbot/worker/package_win32/build/src\gf.c:2154
      From worker 3:    _jl_invoke at /cygdrive/c/buildbot/worker/package_win32/build/src\gf.c:2374 [inlined]
      From worker 3:    ijl_apply_generic at /cygdrive/c/buildbot/worker/package_win32/build/src\gf.c:2575
      From worker 3:    jl_apply at /cygdrive/c/buildbot/worker/package_win32/build/src\julia.h:1852 [inlined]
      From worker 3:    do_apply at /cygdrive/c/buildbot/worker/package_win32/build/src\builtins.c:730
      From worker 3:    jl_f__apply_iterate at /cygdrive/c/buildbot/worker/package_win32/build/src\builtins.c:738
      From worker 3:    #invokelatest#2 at .\essentials.jl:812
      From worker 3:    jl_fptr_args at /cygdrive/c/buildbot/worker/package_win32/build/src\gf.c:2154
      From worker 3:    _jl_invoke at /cygdrive/c/buildbot/worker/package_win32/build/src\gf.c:2393 [inlined]
      From worker 3:    ijl_apply_generic at /cygdrive/c/buildbot/worker/package_win32/build/src\gf.c:2575
      From worker 3:    jl_apply at /cygdrive/c/buildbot/worker/package_win32/build/src\julia.h:1852 [inlined]
      From worker 3:    do_apply at /cygdrive/c/buildbot/worker/package_win32/build/src\builtins.c:730
      From worker 3:    jl_f__apply_iterate at /cygdrive/c/buildbot/worker/package_win32/build/src\builtins.c:738
      From worker 3:    invokelatest##kw at .\essentials.jl:807
      From worker 3:    jl_fptr_args at /cygdrive/c/buildbot/worker/package_win32/build/src\gf.c:2154
      From worker 3:    _jl_invoke at /cygdrive/c/buildbot/worker/package_win32/build/src\gf.c:2393 [inlined]
      From worker 3:    ijl_apply_generic at /cygdrive/c/buildbot/worker/package_win32/build/src\gf.c:2575
      From worker 3:    jl_apply at /cygdrive/c/buildbot/worker/package_win32/build/src\julia.h:1852 [inlined]
      From worker 3:    do_apply at /cygdrive/c/buildbot/worker/package_win32/build/src\builtins.c:730
      From worker 3:    jl_f__apply_iterate at /cygdrive/c/buildbot/worker/package_win32/build/src\builtins.c:738
      From worker 3:    #110 at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\Distributed\src\process_messages.jl:285
      From worker 3:    run_work_thunk at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\Distributed\src\process_messages.jl:70
      From worker 3:    macro expansion at C:\buildbot\worker\package_win32\build\usr\share\julia\stdlib\v1.9\Distributed\src\process_messages.jl:285 [inlined]
      From worker 3:    #109 at .\task.jl:501
      From worker 3:    unknown function (ip: 271128a3)
      From worker 3:    _jl_invoke at /cygdrive/c/buildbot/worker/package_win32/build/src\gf.c:2393 [inlined]
      From worker 3:    ijl_apply_generic at /cygdrive/c/buildbot/worker/package_win32/build/src\gf.c:2575
      From worker 3:    jl_apply at /cygdrive/c/buildbot/worker/package_win32/build/src\julia.h:1852 [inlined]
      From worker 3:    start_task at /cygdrive/c/buildbot/worker/package_win32/build/src\task.c:1085
      From worker 3:    Allocations: 206387379 (Pool: 206277521; Big: 109858); GC: 801
LinearAlgebra/qr                                 (3) |         failed at 2022-08-17T06:43:17.578

[ViralBShah]

Would we be able to extract the failing matrix out of RR? Just to know whether this is a system related, or a matrix related failure?

[gbaraldi]

The error is deep in GC code so it might be that.

[Keno]

If it had happened under rr, yes we could have debugged it. Unfortunately, this happened on win32 so no rr available. If we had had core dumps working, we might have gotten those, which might have also given us the matrix, but that's currently broken also.

[ViralBShah]

Do we still have this issue?