search

snowleopard / hadrian

Hadrian: a new build system for the Glasgow Haskell Compiler. Now merged into the GHC tree!
https://gitlab.haskell.org/ghc/ghc/tree/master/hadrian
MIT License
209 stars 37 forks source link

GHC testsuite rules status #669

Closed alpmestan closed 5 years ago

alpmestan commented 6 years ago

I'm creating this issue to track the work on the testsuite rules that I just resumed. Right now, when I run:

hadrian/build.sh -c -j4 --flavour=perf --build-root=_tmp test

I see a lot of failures.

Test results ``` sh Unexpected results from: TEST="Capi_Ctype_001 Capi_Ctype_002 CaseBinderCPR Conversions DeriveNull DocsInHiFile0 DocsInHiFile1 EvalTest HyperStrUse ImpSafeOnly01 ImpSafeOnly02 ImpSafeOnly03 ImpSafeOnly04 ImpSafeOnly05 ImpSafeOnly06 ImpSafeOnly07 ImpSafeOnly08 ImpSafeOnly09 ImpSafeOnly10 IndTypesPerf InlineArrayAlloc InlineByteArrayAlloc InlineCloneArrayAlloc IntegerConversionRules InternalCounters MethSharing NewTyCo Orphans Ppr001 Ppr002 Ppr003 Ppr004 Ppr005 Ppr006 Ppr007 Ppr008 Ppr009 Ppr010 Ppr011 Ppr012 Ppr013 Ppr014 Ppr015 Ppr016 Ppr017 Ppr018 Ppr019 Ppr020 Ppr021 Ppr022 Ppr023 Ppr024 Ppr025 Ppr026 Ppr027 Ppr028 Ppr029 Ppr030 Ppr031 Ppr032 Ppr033 Ppr034 Ppr035 Ppr036 Ppr037 Ppr038 Ppr039 Ppr040 Ppr041 Ppr042 Ppr043 Ppr044 Ppr045 Ppr046 Ppr048 Roles12 T10052 T10083 T10182 T10255 T10268 T10269 T10272 T10276 T10278 T10280 T10294 T10294a T10296a T10296b T10307 T10309 T10312 T10313 T10320 T10354 T10357 T10358 T10359 T10396 T10399 T10408 T10414 T10420 T10458 T10598 T10694 T10728 T10869 T10923 T10955dyn T10989 T11018 T11148 T11155 T11223_link_order_a_b_2_fail T11223_link_order_a_b_succeed T11223_link_order_b_a_2_succeed T11223_link_order_b_a_succeed T11223_simple_duplicate_lib T11223_simple_link T11223_simple_link_lib T11223_simple_unused_duplicate_lib T11244 T11272 T11321 T1133 T11332 T1133A T11389 T11430 T11478 T11489 T11627b T11788 T11798 T12010 T12023 T12370 T12417 T12485 T12485a T12504 T12567a T12600 T12603 T12733 T12877 T1288_ghci T12955 T12962 T12971 T12990 T12996 T13001 T13025 T13050p T13092 T13092b T13092c T13143 T13155 T13156 T13163 T13168 T13191 T13199 T13317 T13340 T13350 T13367 T13392 T13414 T13468 T13543 T13550 T13585 T13604 T13604a T13623 T13702 T13703 T13710 T1372 T13803 T13807 T13914 T13938 T13942 T14028 T1407 T14075 T14140 T14170 T14257 T14289 T14289b T14289c T14306 T14335 T14346 T14434 T14465 T14626 T14695 T14702 T14774 T14815 T149 T14900 T14931 T14934 T14936 T14955 T14978 T15012 T15056 T15188 T15226 T15226a T15261a T15261b T15303 T1750 T1791 T1792_imports T1959 T2014 T2182 T2228 T2276_ghci T2386 T2412 T2507 T2552 T2566 T2578 T2615 T2636 T2740 T2902 T2976 T2991 T3007 T3055 T3103 T3171 T3231 T3234 T3307 T3333 T3364 T3429 T3449 T3586 T3674 T3717 T3736 T3738 T3772 T3823 T3837 T3890 T3924 T3990 T3994 T4003 T4007 T4059 T4138 T4201 T4239 T4240 T4267 T4306 T4334 T4340 T437 T4474a T4474b T4474c T4830 T4850 T4891 T4903 T4908 T4918 T4930 T4945 T4957 T4978 T5113 T5147 T5168 T5198 T5205 T5237 T5252 T5252Take2 T5327 T5363 T5366 T5402 T5423 T5435_dyn_asm T5435_dyn_gcc T5435_v_asm_a T5435_v_asm_b T5435_v_gcc T5442a T5442b T5442c T5442d T5536 T5549 T5558 T5559 T5584 T5594 T5615 T5623 T5658b T5776 T5792 T5881 T5949 T5955 T5996 T6006 T6025 T6037 T6054 T6106 T6145 T6152 T680 T7014 T7022 T703 T7037 T7040_ghci T706 T7060 T7116 T7165 T7171 T7257 T7299 T7321 T7354a T7360 T7438 T7445 T7476 T7563 T7619 T7796 T7797 T7815 T7835 T7859 T7865 T7962 T7963 T7969 T8002 T8011 T8124 T8129 T8184 T8209 T8221 T8227 T8242 T8274 T8308 T8333 T8430 T8449 T8472 T8500 T8601 T8624 T8628 T8639_api T876 T8766 T8832 T8848 T8959a T9032 T9078 T9086 T9203 T9204 T9208 T9263 T9293 T9405 T9509 T9579_outofheap_rtsall T9579_outofheap_rtsall_no_suggestions T9579_outofheap_rtsnone T9579_outofheap_rtssome T9579_stackoverflow_rtsall T9579_stackoverflow_rtsall_no_suggestions T9579_stackoverflow_rtsnone T9579_stackoverflow_rtssome T9905 T9905b T9905fail1 T9905fail2 T9905fail3 T9930fail T9938 T9938B TH_Depends TH_spliceE5_prof TH_spliceE5_prof_ext Tc267 Tc271 annotations apirecomp001 atomicxchg base01 bkpcabal01 bkpcabal02 bkpcabal03 bkpcabal04 bkpcabal05 bkpcabal06 bkpcabal07 boolFormula break001 break005 break006 break026 break027 bug1465 bug1677 bundle-export cabal01 cabal03 cabal04 cabal05 cabal06 cabal08 cabal09 comments compact_append compact_autoexpand compact_bench compact_cycle compact_function compact_gc compact_huge_array compact_largemap compact_loop compact_mutable compact_pinned compact_serialize compact_share compact_simple compact_simple_array compact_threads conc069 conc069a conc070 conc072 concio001 concio001.thr conflicting_flags debug determ002 determ003 determ005 determ006 determ007 determ008 determ009 determ010 determ011 determ012 determ013 determ014 determ015 determ016 determ017 determ018 determ019 determ021 determ022 driver011 driver012 driver013 driver014 driver015 driver016 driver017 driver018 driver018a driver019 driver021 driver022 driver023 driver024 driver024a driver025 driver026 driver027 driver028 driver031 driver032 driver033 driver034 driver035 driver041 driver042 driver042stub driver043 driver044 driver045 driver051 driver052 driver053 driver061a driver061b driver062a driver062b driver062c driver062d driver062e driver063 driver064 driver065 driver066 driver067 driver070 driver071 driver081a driver081b driver100 driver200 drvfail016 dynbrk002 environment001 exampleTest exec_signals ffi002 ffi014 ffi018_ghci ffi023 ffishutdown forkprocess01 fromToInteger gadt17 gadt23 ghc-e-fail1 ghc-e-fail2 ghc-e001 ghc-e002 ghc-e003 ghc-e004 ghc-e005 ghc-e006 ghci037 ghci055 ghci056 ghci057 ghcilink001 ghcilink002 ghcilink003 ghcilink004 ghcilink005 ghcilink006 ghciprog004 ghcpkg01 ghcpkg02 ghcpkg03 ghcpkg04 ghcpkg05 ghcpkg06 ghcpkg07 haddock.Cabal haddock.base haddock.compiler hist001 hist002 hpc_ghc_ghci hsc2hs001 hsc2hs002 hsc2hs003 hsc2hs004 inline-check integerConstantFolding join004 join007 layout001 layout002 layout003 layout004 layout005 layout006 layout007 layout008 layout009 lazy-bs-alloc linker_error1 linker_error2 linker_error3 linker_unload linkwhole listcomps literals load-main load_short_name memo001 memo002 mod144 mod146 mod158 mod160 mod165 mod166 mod167 mod175 mode001 numa001 numsparks001 nursery-chunks1 outofmem outofmem2 parseTree parsed plugin-recomp-change plugin-recomp-change-prof plugin-recomp-flags plugin-recomp-impure plugin-recomp-pure plugins01 plugins02 plugins03 plugins07 plugins08 plugins09 plugins10 plugins11 plugins12 plugins13 plugins14 plugins15 print001 print002 print003 print004 print005 print006 print008 print010 print012 print013 print014 print016 print017 print023 print024 print028 process007 prof-doc-fib prof-doc-last profinline001 prog014 read041 recomp003 recomp004 recomp009 recomp010 recomp011 recomp012 recomp013 recomp015 recomp016 retc001 retc003 rn.prog006 rtsopts001 rtsopts002 safePkg01 scc001 scc002 scc003 scc005 shadow showsrcspan simpl021 space_leak_001 spec-inline stm056 str-rules sum_mod tc170 tc173 tc245 tc266 testwsdeque withRtsOpts write_interface_make write_interface_oneshot" SUMMARY for test run started at Thu Aug 30 19:10:39 2018 CEST 0:21:40 spent to go through 6527 total tests, which gave rise to 24015 test cases, of which 17508 were skipped 27 had missing libraries 5635 expected passes 96 expected failures 56 caused framework failures 0 caused framework warnings 1 unexpected passes 704 unexpected failures 43 unexpected stat failures Unexpected passes: testsuite/tests/stranal/should_compile/T9208.run T9208 [unexpected] (optasm) Unexpected failures: testsuite/tests/backpack/cabal/bkpcabal01/bkpcabal01.run bkpcabal01 [bad exit code] (normal) testsuite/tests/backpack/cabal/bkpcabal02/bkpcabal02.run bkpcabal02 [bad exit code] (normal) testsuite/tests/backpack/cabal/bkpcabal03/bkpcabal03.run bkpcabal03 [bad exit code] (normal) testsuite/tests/backpack/cabal/bkpcabal04/bkpcabal04.run bkpcabal04 [bad exit code] (normal) testsuite/tests/backpack/cabal/bkpcabal05/bkpcabal05.run bkpcabal05 [bad exit code] (normal) testsuite/tests/backpack/cabal/bkpcabal06/bkpcabal06.run bkpcabal06 [bad exit code] (normal) testsuite/tests/backpack/cabal/bkpcabal07/bkpcabal07.run bkpcabal07 [bad exit code] (normal) testsuite/tests/cabal/ghcpkg01.run ghcpkg01 [bad exit code] (normal) testsuite/tests/cabal/ghcpkg02.run ghcpkg02 [bad exit code] (normal) testsuite/tests/cabal/ghcpkg03.run ghcpkg03 [bad exit code] (normal) testsuite/tests/cabal/ghcpkg04.run ghcpkg04 [bad exit code] (normal) testsuite/tests/cabal/ghcpkg05.run ghcpkg05 [bad exit code] (normal) testsuite/tests/cabal/ghcpkg06.run ghcpkg06 [bad exit code] (normal) testsuite/tests/cabal/ghcpkg07.run ghcpkg07 [bad exit code] (normal) testsuite/tests/cabal/T1750.run T1750 [bad exit code] (normal) testsuite/tests/cabal/T5442a.run T5442a [bad exit code] (normal) testsuite/tests/cabal/T5442b.run T5442b [bad exit code] (normal) testsuite/tests/cabal/T5442c.run T5442c [bad exit code] (normal) testsuite/tests/cabal/T5442d.run T5442d [bad exit code] (normal) testsuite/tests/cabal/shadow.run shadow [bad exit code] (normal) testsuite/tests/cabal/T12485a.run T12485a [bad exit code] (normal) testsuite/tests/cabal/T13703.run T13703 [bad exit code] (normal) testsuite/tests/cabal/T12485/T12485.run T12485 [bad exit code] (normal) testsuite/tests/cabal/T12733/T12733.run T12733 [bad exit code] (normal) testsuite/tests/cabal/cabal01/cabal01.run cabal01 [bad exit code] (normal) testsuite/tests/cabal/cabal03/cabal03.run cabal03 [bad exit code] (normal) testsuite/tests/cabal/cabal04/cabal04.run cabal04 [bad exit code] (normal) testsuite/tests/cabal/cabal05/cabal05.run cabal05 [bad exit code] (normal) testsuite/tests/cabal/cabal06/cabal06.run cabal06 [bad exit code] (normal) testsuite/tests/cabal/cabal08/cabal08.run cabal08 [bad exit code] (normal) testsuite/tests/cabal/cabal09/cabal09.run cabal09 [bad exit code] (normal) testsuite/tests/cmm/opt/T15188.run T15188 [bad exit code] (normal) testsuite/tests/codeGen/should_compile/T2578.run T2578 [bad exit code] (normal) testsuite/tests/codeGen/should_compile/debug.run debug [bad exit code] (normal) testsuite/tests/codeGen/should_compile/T14626.run T14626 [bad exit code] (normal) testsuite/tests/codeGen/should_run/T10414.run T10414 [exit code non-0] (threaded2) testsuite/tests/codeGen/should_run/T14346.run T14346 [exit code non-0] (threaded1) testsuite/tests/concurrent/should_run/conc069.run conc069 [bad exit code] (ghci) testsuite/tests/concurrent/should_run/conc069a.run conc069a [bad exit code] (ghci) testsuite/tests/concurrent/should_run/conc070.run conc070 [bad stdout] (ghci) testsuite/tests/concurrent/should_run/conc072.run conc072 [exit code non-0] (threaded2) testsuite/tests/concurrent/should_run/T3429.run T3429 [exit code non-0] (debug_numa) testsuite/tests/concurrent/should_run/numsparks001.run numsparks001 [exit code non-0] (threaded1) testsuite/tests/concurrent/should_run/T5558.run T5558 [exit code non-0] (threaded2) testsuite/tests/deSugar/should_compile/T5252.run T5252 [bad exit code] (normal) testsuite/tests/deSugar/should_compile/T5252Take2.run T5252Take2 [bad exit code] (normal) testsuite/tests/deSugar/should_compile/T14815.run T14815 [bad exit code] (normal) testsuite/tests/dependent/should_compile/T13938.run T13938 [bad exit code] (normal) testsuite/tests/deriving/should_compile/T1133.run T1133 [bad exit code] (normal) testsuite/tests/deriving/should_compile/T11148.run T11148 [bad exit code] (normal) testsuite/tests/deriving/should_fail/drvfail016.run drvfail016 [bad exit code] (normal) testsuite/tests/deriving/should_fail/T1133A.run T1133A [bad exit code] (normal) testsuite/tests/determinism/T13807/T13807.run T13807 [bad exit code] (normal) testsuite/tests/determinism/determ002/determ002.run determ002 [bad exit code] (normal) testsuite/tests/determinism/determ003/determ003.run determ003 [bad exit code] (normal) testsuite/tests/determinism/determ005/determ005.run determ005 [bad exit code] (normal) testsuite/tests/determinism/determ006/determ006.run determ006 [bad exit code] (normal) testsuite/tests/determinism/determ007/determ007.run determ007 [bad exit code] (normal) testsuite/tests/determinism/determ008/determ008.run determ008 [bad exit code] (normal) testsuite/tests/determinism/determ009/determ009.run determ009 [bad exit code] (normal) testsuite/tests/determinism/determ010/determ010.run determ010 [bad exit code] (normal) testsuite/tests/determinism/determ011/determ011.run determ011 [bad exit code] (normal) testsuite/tests/determinism/determ012/determ012.run determ012 [bad exit code] (normal) testsuite/tests/determinism/determ013/determ013.run determ013 [bad exit code] (normal) testsuite/tests/determinism/determ014/determ014.run determ014 [bad exit code] (normal) testsuite/tests/determinism/determ015/determ015.run determ015 [bad exit code] (normal) testsuite/tests/determinism/determ016/determ016.run determ016 [bad exit code] (normal) testsuite/tests/determinism/determ017/determ017.run determ017 [bad exit code] (normal) testsuite/tests/determinism/determ018/determ018.run determ018 [bad exit code] (normal) testsuite/tests/determinism/determ019/determ019.run determ019 [bad exit code] (normal) testsuite/tests/determinism/determ021/determ021.run determ021 [bad exit code] (normal) testsuite/tests/determinism/determ022/determ022.run determ022 [bad exit code] (normal) testsuite/tests/driver/driver011.run driver011 [bad exit code] (normal) testsuite/tests/driver/driver012.run driver012 [bad exit code] (normal) testsuite/tests/driver/driver013.run driver013 [bad exit code] (normal) testsuite/tests/driver/driver014.run driver014 [bad exit code] (normal) testsuite/tests/driver/driver015.run driver015 [bad exit code] (normal) testsuite/tests/driver/driver016.run driver016 [bad exit code] (normal) testsuite/tests/driver/driver017.run driver017 [bad exit code] (normal) testsuite/tests/driver/driver018.run driver018 [bad exit code] (normal) testsuite/tests/driver/driver018a.run driver018a [bad exit code] (normal) testsuite/tests/driver/driver022.run driver022 [bad exit code] (normal) testsuite/tests/driver/driver019.run driver019 [bad exit code] (normal) testsuite/tests/driver/driver021.run driver021 [bad exit code] (normal) testsuite/tests/driver/driver023.run driver023 [bad exit code] (normal) testsuite/tests/driver/driver024a.run driver024a [bad exit code] (normal) testsuite/tests/driver/driver025.run driver025 [bad exit code] (normal) testsuite/tests/driver/driver024.run driver024 [bad exit code] (normal) testsuite/tests/driver/driver026.run driver026 [bad exit code] (normal) testsuite/tests/driver/driver032.run driver032 [bad exit code] (normal) testsuite/tests/driver/driver027.run driver027 [bad exit code] (normal) testsuite/tests/driver/driver028.run driver028 [bad exit code] (normal) testsuite/tests/driver/driver031.run driver031 [bad exit code] (normal) testsuite/tests/driver/driver033.run driver033 [bad exit code] (normal) testsuite/tests/driver/driver034.run driver034 [bad exit code] (normal) testsuite/tests/driver/driver041.run driver041 [bad exit code] (normal) testsuite/tests/driver/driver035.run driver035 [bad exit code] (normal) testsuite/tests/driver/driver042.run driver042 [bad exit code] (normal) testsuite/tests/driver/driver042stub.run driver042stub [bad exit code] (normal) testsuite/tests/driver/driver043.run driver043 [bad exit code] (normal) testsuite/tests/driver/driver044.run driver044 [bad exit code] (normal) testsuite/tests/driver/driver045.run driver045 [bad exit code] (normal) testsuite/tests/driver/driver051.run driver051 [bad exit code] (normal) testsuite/tests/driver/driver052.run driver052 [bad exit code] (normal) testsuite/tests/driver/driver053.run driver053 [bad exit code] (normal) testsuite/tests/driver/driver061a.run driver061a [bad exit code] (normal) testsuite/tests/driver/driver062a.run driver062a [bad exit code] (normal) testsuite/tests/driver/driver062b.run driver062b [bad exit code] (normal) testsuite/tests/driver/driver061b.run driver061b [bad exit code] (normal) testsuite/tests/driver/driver062c.run driver062c [bad exit code] (normal) testsuite/tests/driver/driver062d.run driver062d [bad exit code] (normal) testsuite/tests/driver/driver062e.run driver062e [bad exit code] (normal) testsuite/tests/driver/driver063.run driver063 [bad exit code] (normal) testsuite/tests/driver/driver064.run driver064 [bad exit code] (normal) testsuite/tests/driver/driver066.run driver066 [bad exit code] (normal) testsuite/tests/driver/driver065.run driver065 [bad exit code] (normal) testsuite/tests/driver/driver067.run driver067 [bad exit code] (normal) testsuite/tests/driver/driver070.run driver070 [bad exit code] (normal) testsuite/tests/driver/driver071.run driver071 [bad exit code] (normal) testsuite/tests/driver/driver081a.run driver081a [bad exit code] (normal) testsuite/tests/driver/driver081b.run driver081b [bad exit code] (normal) testsuite/tests/driver/driver100.run driver100 [bad exit code] (normal) testsuite/tests/driver/T2566.run T2566 [bad exit code] (normal) testsuite/tests/driver/driver200.run driver200 [bad exit code] (normal) testsuite/tests/driver/mode001.run mode001 [bad exit code] (normal) testsuite/tests/driver/T3674.run T3674 [bad exit code] (normal) testsuite/tests/driver/rtsopts001.run rtsopts001 [bad exit code] (normal) testsuite/tests/driver/rtsopts002.run rtsopts002 [bad exit code] (normal) testsuite/tests/driver/withRtsOpts.run withRtsOpts [bad exit code] (normal) testsuite/tests/driver/T3364.run T3364 [bad exit code] (normal) testsuite/tests/driver/T706.run T706 [bad exit code] (normal) testsuite/tests/driver/T5584.run T5584 [bad exit code] (normal) testsuite/tests/driver/T5198.run T5198 [bad exit code] (normal) testsuite/tests/driver/T7060.run T7060 [bad exit code] (normal) testsuite/tests/driver/T6037.run T6037 [bad exit code] (normal) testsuite/tests/driver/T7563.run T7563 [bad exit code] (normal) testsuite/tests/driver/T2507.run T2507 [bad exit code] (normal) testsuite/tests/driver/T703.run T703 [bad exit code] (normal) testsuite/tests/driver/T8959a.run T8959a [bad exit code] (normal) testsuite/tests/driver/T2182.run T2182 [bad exit code] (normal) testsuite/tests/driver/write_interface_oneshot.run write_interface_oneshot [bad exit code] (normal) testsuite/tests/driver/write_interface_make.run write_interface_make [bad exit code] (normal) testsuite/tests/driver/T9938.run T9938 [bad exit code] (normal) testsuite/tests/driver/T9938B.run T9938B [bad exit code] (normal) testsuite/tests/driver/T10182.run T10182 [bad exit code] (normal) testsuite/tests/driver/T10869.run T10869 [bad exit code] (normal) testsuite/tests/driver/T10320.run T10320 [bad exit code] (normal) testsuite/tests/driver/T10923.run T10923 [bad exit code] (normal) testsuite/tests/driver/T12971.run T12971 [bad exit code] (normal) testsuite/tests/driver/T12955.run T12955 [bad exit code] (normal) testsuite/tests/driver/T13604.run T13604 [bad exit code] (normal) testsuite/tests/driver/T13604a.run T13604a [bad exit code] (normal) testsuite/tests/driver/inline-check.run inline-check [stderr mismatch] (normal) testsuite/tests/driver/T13392/T13392.run T13392 [bad exit code] (normal) testsuite/tests/driver/T13710/T13710.run T13710 [bad exit code] (normal) testsuite/tests/driver/T1372/T1372.run T1372 [bad exit code] (normal) testsuite/tests/driver/T13803/T13803.run T13803 [bad exit code] (normal) testsuite/tests/driver/T13914/T13914.run T13914 [bad exit code] (normal) testsuite/tests/driver/T14075/T14075.run T14075 [bad exit code] (normal) testsuite/tests/driver/T1959/T1959.run T1959 [bad exit code] (normal) testsuite/tests/driver/T3007/T3007.run T3007 [bad exit code] (normal) testsuite/tests/driver/T437/T437.run T437 [bad exit code] (normal) testsuite/tests/driver/T5147/T5147.run T5147 [bad exit code] (normal) testsuite/tests/driver/T7835/T7835.run T7835 [bad exit code] (normal) testsuite/tests/driver/T8184/T8184.run T8184 [bad exit code] (normal) testsuite/tests/driver/bug1677/bug1677.run bug1677 [bad exit code] (normal) testsuite/tests/driver/conflicting_flags/conflicting_flags.run conflicting_flags [bad exit code] (normal) testsuite/tests/driver/linkwhole/linkwhole.run linkwhole [bad exit code] (normal) testsuite/tests/driver/recomp003/recomp003.run recomp003 [bad exit code] (normal) testsuite/tests/driver/recomp004/recomp004.run recomp004 [bad exit code] (normal) testsuite/tests/driver/recomp009/recomp009.run recomp009 [bad exit code] (normal) testsuite/tests/driver/recomp010/recomp010.run recomp010 [bad exit code] (normal) testsuite/tests/driver/recomp011/recomp011.run recomp011 [bad exit code] (normal) testsuite/tests/driver/recomp012/recomp012.run recomp012 [bad exit code] (normal) testsuite/tests/driver/recomp013/recomp013.run recomp013 [bad exit code] (normal) testsuite/tests/driver/recomp015/recomp015.run recomp015 [bad exit code] (normal) testsuite/tests/driver/recomp016/recomp016.run recomp016 [bad exit code] (normal) testsuite/tests/driver/retc001/retc001.run retc001 [bad exit code] (normal) testsuite/tests/driver/retc003/retc003.run retc003 [bad exit code] (normal) testsuite/tests/dynlibs/T13702.run T13702 [bad exit code] (normal) testsuite/tests/ffi/should_run/ffi002.run ffi002 [exit code non-0] (normal) testsuite/tests/ffi/should_run/ffi014.run ffi014 [exit code non-0] (threaded1) testsuite/tests/ffi/should_run/ffi018_ghci.run ffi018_ghci [bad stderr] (ghci) testsuite/tests/ffi/should_run/T1288_ghci.run T1288_ghci [bad stderr] (ghci) testsuite/tests/ffi/should_run/T2276_ghci.run T2276_ghci [bad stderr] (ghci) testsuite/tests/ffi/should_run/T5402.run T5402 [exit code non-0] (normal) testsuite/tests/ffi/should_run/T5594.run T5594 [exit code non-0] (normal) testsuite/tests/ffi/should_run/Capi_Ctype_002.run Capi_Ctype_002 [bad exit code] (normal) testsuite/tests/ffi/should_run/Capi_Ctype_001.run Capi_Ctype_001 [bad exit code] (normal) testsuite/tests/ffi/should_run/ffi023.run ffi023 [exit code non-0] (normal) testsuite/tests/gadt/gadt17.run gadt17 [bad exit code] (normal) testsuite/tests/gadt/gadt23.run gadt23 [bad exit code] (normal) testsuite/tests/gadt/T7321.run T7321 [bad exit code] (normal) testsuite/tests/generics/T15012.run T15012 [bad exit code] (normal) testsuite/tests/ghc-api/T6145.run T6145 [bad exit code] (normal) testsuite/tests/ghc-api/T8639_api.run T8639_api [bad exit code] (normal) testsuite/tests/ghc-api/T8628.run T8628 [bad exit code] (normal) testsuite/tests/ghc-api/T10052/T10052.run T10052 [bad exit code] (normal) testsuite/tests/ghc-api/T4891/T4891.run T4891 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/parseTree.run parseTree [bad exit code] (normal) testsuite/tests/ghc-api/annotations/annotations.run annotations [bad exit code] (normal) testsuite/tests/ghc-api/annotations/comments.run comments [bad exit code] (normal) testsuite/tests/ghc-api/annotations/exampleTest.run exampleTest [bad exit code] (normal) testsuite/tests/ghc-api/annotations/listcomps.run listcomps [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10255.run T10255 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10268.run T10268 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10269.run T10269 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10280.run T10280 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10312.run T10312 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10307.run T10307 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10309.run T10309 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/boolFormula.run boolFormula [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10357.run T10357 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10358.run T10358 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10278.run T10278 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10354.run T10354 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10396.run T10396 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10399.run T10399 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10313.run T10313 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T11018.run T11018 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/bundle-export.run bundle-export [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10598.run T10598 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10276.run T10276 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T11321.run T11321 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T11332.run T11332 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T11430.run T11430 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/load-main.run load-main [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T12417.run T12417 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T13163.run T13163 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T15303.run T15303 [bad exit code] (normal) testsuite/tests/ghc-api/annotations-literals/literals.run literals [bad exit code] (normal) testsuite/tests/ghc-api/annotations-literals/parsed.run parsed [bad exit code] (normal) testsuite/tests/ghc-api/apirecomp001/apirecomp001.run apirecomp001 [bad exit code] (normal) testsuite/tests/ghc-api/show-srcspan/showsrcspan.run showsrcspan [bad exit code] (normal) testsuite/tests/ghc-e/should_fail/T7962.run T7962 [bad exit code] (normal) testsuite/tests/ghc-e/should_fail/T9905fail1.run T9905fail1 [bad exit code] (normal) testsuite/tests/ghc-e/should_fail/T9905fail2.run T9905fail2 [bad exit code] (normal) testsuite/tests/ghc-e/should_fail/T9905fail3.run T9905fail3 [bad exit code] (normal) testsuite/tests/ghc-e/should_fail/ghc-e-fail1.run ghc-e-fail1 [bad exit code] (normal) testsuite/tests/ghc-e/should_fail/ghc-e-fail2.run ghc-e-fail2 [bad exit code] (normal) testsuite/tests/ghc-e/should_fail/T9930fail.run T9930fail [bad exit code] (normal) testsuite/tests/ghc-e/should_run/ghc-e001.run ghc-e001 [bad exit code] (normal) testsuite/tests/ghc-e/should_run/ghc-e002.run ghc-e002 [bad exit code] (normal) testsuite/tests/ghc-e/should_run/ghc-e003.run ghc-e003 [bad exit code] (normal) testsuite/tests/ghc-e/should_run/ghc-e004.run ghc-e004 [bad exit code] (normal) testsuite/tests/ghc-e/should_run/ghc-e005.run ghc-e005 [bad exit code] (normal) testsuite/tests/ghc-e/should_run/ghc-e006.run ghc-e006 [bad exit code] (normal) testsuite/tests/ghc-e/should_run/T2228.run T2228 [bad exit code] (normal) testsuite/tests/ghc-e/should_run/T2636.run T2636 [bad exit code] (normal) testsuite/tests/ghc-e/should_run/T3890.run T3890 [bad exit code] (normal) testsuite/tests/ghc-e/should_run/T7299.run T7299 [bad exit code] (normal) testsuite/tests/ghc-e/should_run/T9086.run T9086 [bad exit code] (normal) testsuite/tests/ghc-e/should_run/T9905.run T9905 [bad exit code] (normal) testsuite/tests/ghc-e/should_run/T9905b.run T9905b [bad exit code] (normal) testsuite/tests/ghc-e/should_run/T11478.run T11478 [bad exit code] (normal) testsuite/tests/ghci/linking/ghcilink001.run ghcilink001 [bad exit code] (normal) testsuite/tests/ghci/linking/ghcilink003.run ghcilink003 [bad exit code] (normal) testsuite/tests/ghci/linking/ghcilink002.run ghcilink002 [bad exit code] (normal) testsuite/tests/ghci/linking/ghcilink004.run ghcilink004 [bad exit code] (normal) testsuite/tests/ghci/linking/ghcilink005.run ghcilink005 [bad exit code] (normal) testsuite/tests/ghci/linking/ghcilink006.run ghcilink006 [bad exit code] (normal) testsuite/tests/ghci/linking/T3333.run T3333 [bad exit code] (normal) testsuite/tests/ghci/linking/dyn/load_short_name.run load_short_name [bad exit code] (normal) testsuite/tests/ghci/linking/dyn/T10955dyn.run T10955dyn [bad exit code] (normal) testsuite/tests/ghci/linking/dyn/T1407.run T1407 [bad exit code] (normal) testsuite/tests/ghci/linking/dyn/T10458.run T10458 [bad exit code] (ghci) testsuite/tests/ghci/prog004/ghciprog004.run ghciprog004 [bad exit code] (normal) testsuite/tests/ghci/scripts/ghci037.run ghci037 [bad exit code] (normal) testsuite/tests/ghci/scripts/ghci055.run ghci055 [bad stdout] (ghci) testsuite/tests/ghci/scripts/ghci057.run ghci057 [bad stdout] (ghci) testsuite/tests/ghci/scripts/ghci056.run ghci056 [bad stderr] (ghci) testsuite/tests/ghci/scripts/T2976.run T2976 [bad stdout] (ghci) testsuite/tests/ghci/scripts/T6106.run T6106 [bad stderr] (ghci) testsuite/tests/ghci/scripts/T9293.run T9293 [bad stdout] (ghci) testsuite/tests/ghci/scripts/T10408.run T10408 [bad exit code] (normal) testsuite/tests/ghci/scripts/T10989.run T10989 [bad stderr] (ghci) testsuite/tests/ghci/scripts/T11389.run T11389 [bad exit code] (normal) testsuite/tests/ghci/scripts/T12023.run T12023 [bad exit code] (normal) testsuite/tests/ghci.debugger/scripts/print001.run print001 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print002.run print002 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print001.run print001 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print003.run print003 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print002.run print002 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print004.run print004 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print003.run print003 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print005.run print005 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print004.run print004 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print006.run print006 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print005.run print005 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print006.run print006 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print008.run print008 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print008.run print008 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print010.run print010 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print012.run print012 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print010.run print010 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print012.run print012 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print013.run print013 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print014.run print014 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print016.run print016 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print013.run print013 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print014.run print014 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print017.run print017 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print016.run print016 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print017.run print017 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print023.run print023 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print024.run print024 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print023.run print023 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print024.run print024 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print028.run print028 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print028.run print028 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/break001.run break001 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/break001.run break001 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/break005.run break005 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/break006.run break006 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/break005.run break005 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/break006.run break006 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/break026.run break026 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/break027.run break027 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/break026.run break026 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/break027.run break027 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/dynbrk002.run dynbrk002 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/dynbrk002.run dynbrk002 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/hist001.run hist001 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/hist002.run hist002 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/T2740.run T2740 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/hist001.run hist001 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/hist002.run hist002 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/T2740.run T2740 [bad stdout] (ghci-ext) testsuite/tests/hpc/T2991.run T2991 [exit code non-0] (hpc) testsuite/tests/hpc/T11798.run T11798 [bad exit code] (normal) testsuite/tests/hsc2hs/hsc2hs001.run hsc2hs001 [bad exit code] (normal) testsuite/tests/hsc2hs/hsc2hs002.run hsc2hs002 [bad exit code] (normal) testsuite/tests/hsc2hs/hsc2hs003.run hsc2hs003 [bad exit code] (normal) testsuite/tests/hsc2hs/hsc2hs004.run hsc2hs004 [bad exit code] (normal) testsuite/tests/hsc2hs/T3837.run T3837 [bad exit code] (normal) testsuite/tests/hsc2hs/T4340.run T4340 [bad exit code] (normal) testsuite/tests/hsc2hs/T10272.run T10272 [bad exit code] (normal) testsuite/tests/hsc2hs/T12504.run T12504 [bad exit code] (normal) testsuite/tests/indexed-types/should_compile/NewTyCo.run NewTyCo [bad exit code] (normal) testsuite/tests/indexed-types/should_compile/IndTypesPerf.run IndTypesPerf [bad exit code] (normal) testsuite/tests/indexed-types/should_compile/T5955.run T5955 [bad exit code] (normal) testsuite/tests/indexed-types/should_compile/T6152.run T6152 [bad exit code] (normal) testsuite/tests/indexed-types/should_compile/T8002.run T8002 [bad exit code] (normal) testsuite/tests/indexed-types/should_compile/T8011.run T8011 [bad exit code] (normal) testsuite/tests/indexed-types/should_compile/T8500.run T8500 [bad exit code] (normal) testsuite/tests/indexed-types/should_compile/T13092b/T13092b.run T13092b [bad exit code] (normal) testsuite/tests/indexed-types/should_fail/T7354a.run T7354a [bad exit code] (normal) testsuite/tests/indexed-types/should_fail/T8227.run T8227 [bad exit code] (normal) testsuite/tests/indexed-types/should_fail/T8129.run T8129 [bad exit code] (normal) testsuite/tests/indexed-types/should_fail/T13092/T13092.run T13092 [bad exit code] (normal) testsuite/tests/indexed-types/should_fail/T13092c/T13092c.run T13092c [bad exit code] (normal) testsuite/tests/layout/layout001.run layout001 [bad exit code] (normal) testsuite/tests/layout/layout002.run layout002 [bad exit code] (normal) testsuite/tests/layout/layout003.run layout003 [bad exit code] (normal) testsuite/tests/layout/layout004.run layout004 [bad exit code] (normal) testsuite/tests/layout/layout005.run layout005 [bad exit code] (normal) testsuite/tests/layout/layout006.run layout006 [bad exit code] (normal) testsuite/tests/layout/layout007.run layout007 [bad exit code] (normal) testsuite/tests/layout/layout009.run layout009 [bad exit code] (normal) testsuite/tests/layout/layout008.run layout008 [bad exit code] (normal) testsuite/tests/lib/integer/integerConstantFolding.run integerConstantFolding [bad exit code] (normal) testsuite/tests/lib/integer/fromToInteger.run fromToInteger [bad exit code] (normal) testsuite/tests/lib/integer/IntegerConversionRules.run IntegerConversionRules [bad exit code] (normal) testsuite/tests/module/mod144.run mod144 [bad exit code] (normal) testsuite/tests/module/mod146.run mod146 [bad exit code] (normal) testsuite/tests/module/mod158.run mod158 [bad exit code] (normal) testsuite/tests/module/mod160.run mod160 [bad exit code] (normal) testsuite/tests/module/mod165.run mod165 [bad exit code] (normal) testsuite/tests/module/mod166.run mod166 [bad exit code] (normal) testsuite/tests/module/mod167.run mod167 [bad exit code] (normal) testsuite/tests/module/base01/base01.run base01 [bad exit code] (normal) testsuite/tests/module/mod175/mod175.run mod175 [bad exit code] (normal) testsuite/tests/numeric/should_compile/T7116.run T7116 [bad exit code] (normal) testsuite/tests/numeric/should_compile/T14170.run T14170 [bad exit code] (normal) testsuite/tests/numeric/should_compile/T14465.run T14465 [bad exit code] (normal) testsuite/tests/numeric/should_run/T7014.run T7014 [bad exit code] (normal) testsuite/tests/parser/should_compile/read041.run read041 [exit code non-0] (normal) testsuite/tests/parser/should_compile/T7476/T7476.run T7476 [bad exit code] (normal) testsuite/tests/parser/should_fail/T8430.run T8430 [stderr mismatch] (normal) testsuite/tests/parser/should_fail/T13414.run T13414 [stderr mismatch] (normal) testsuite/tests/patsyn/should_compile/T13350/T13350.run T13350 [bad exit code] (normal) testsuite/tests/perf/compiler/T4007.run T4007 [bad exit code] (normal) testsuite/tests/perf/haddock/haddock.base.run haddock.base [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.base.run/haddock.t'] (normal) testsuite/tests/perf/haddock/haddock.Cabal.run haddock.Cabal [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.Cabal.run/haddock.t'] (normal) testsuite/tests/perf/haddock/haddock.compiler.run haddock.compiler [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.compiler.run/haddock.t'] (normal) testsuite/tests/perf/should_run/T3736.run T3736 [bad exit code] (normal) testsuite/tests/perf/should_run/T2902.run T2902 [bad exit code] (normal) testsuite/tests/perf/should_run/T149.run T149 [bad exit code] (normal) testsuite/tests/plugins/plugins01.run plugins01 [bad exit code] (normal) testsuite/tests/plugins/plugins02.run plugins02 [stderr mismatch] (normal) testsuite/tests/plugins/plugins03.run plugins03 [stderr mismatch] (normal) testsuite/tests/plugins/plugins07.run plugins07 [bad exit code] (normal) testsuite/tests/plugins/plugins08.run plugins08 [bad exit code] (normal) testsuite/tests/plugins/plugins09.run plugins09 [bad exit code] (normal) testsuite/tests/plugins/plugins11.run plugins11 [bad exit code] (normal) testsuite/tests/plugins/plugins12.run plugins12 [bad exit code] (normal) testsuite/tests/plugins/plugins13.run plugins13 [bad exit code] (normal) testsuite/tests/plugins/plugins14.run plugins14 [bad exit code] (normal) testsuite/tests/plugins/plugins15.run plugins15 [bad exit code] (normal) testsuite/tests/plugins/T10420.run T10420 [bad exit code] (normal) testsuite/tests/plugins/T10294.run T10294 [bad exit code] (normal) testsuite/tests/plugins/T10294a.run T10294a [bad exit code] (normal) testsuite/tests/plugins/T11244.run T11244 [bad exit code] (normal) testsuite/tests/plugins/T12567a.run T12567a [bad exit code] (normal) testsuite/tests/plugins/plugin-recomp-pure.run plugin-recomp-pure [bad exit code] (normal) testsuite/tests/plugins/T14335.run T14335 [stderr mismatch] (normal) testsuite/tests/plugins/plugin-recomp-flags.run plugin-recomp-flags [bad exit code] (normal) testsuite/tests/plugins/plugin-recomp-impure.run plugin-recomp-impure [bad exit code] (normal) testsuite/tests/plugins/plugin-recomp-change.run plugin-recomp-change [bad exit code] (normal) testsuite/tests/plugins/plugin-recomp-change-prof.run plugin-recomp-change-prof [bad exit code] (normal) testsuite/tests/polykinds/T5881.run T5881 [bad exit code] (normal) testsuite/tests/polykinds/T6025.run T6025 [bad exit code] (normal) testsuite/tests/polykinds/T6054.run T6054 [bad exit code] (normal) testsuite/tests/polykinds/T7022.run T7022 [bad exit code] (normal) testsuite/tests/polykinds/T7438.run T7438 [bad exit code] (normal) testsuite/tests/polykinds/T8449.run T8449 [bad exit code] (normal) testsuite/tests/polykinds/T9263.run T9263 [bad exit code] (normal) testsuite/tests/printer/Ppr001.run Ppr001 [bad exit code] (normal) testsuite/tests/printer/Ppr002.run Ppr002 [bad exit code] (normal) testsuite/tests/printer/Ppr003.run Ppr003 [bad exit code] (normal) testsuite/tests/printer/Ppr004.run Ppr004 [bad exit code] (normal) testsuite/tests/printer/Ppr005.run Ppr005 [bad exit code] (normal) testsuite/tests/printer/Ppr006.run Ppr006 [bad exit code] (normal) testsuite/tests/printer/Ppr007.run Ppr007 [bad exit code] (normal) testsuite/tests/printer/Ppr008.run Ppr008 [bad exit code] (normal) testsuite/tests/printer/Ppr009.run Ppr009 [bad exit code] (normal) testsuite/tests/printer/Ppr010.run Ppr010 [bad exit code] (normal) testsuite/tests/printer/Ppr011.run Ppr011 [bad exit code] (normal) testsuite/tests/printer/Ppr012.run Ppr012 [bad exit code] (normal) testsuite/tests/printer/Ppr013.run Ppr013 [bad exit code] (normal) testsuite/tests/printer/Ppr014.run Ppr014 [bad exit code] (normal) testsuite/tests/printer/Ppr015.run Ppr015 [bad exit code] (normal) testsuite/tests/printer/Ppr016.run Ppr016 [bad exit code] (normal) testsuite/tests/printer/Ppr017.run Ppr017 [bad exit code] (normal) testsuite/tests/printer/Ppr018.run Ppr018 [bad exit code] (normal) testsuite/tests/printer/Ppr019.run Ppr019 [bad exit code] (normal) testsuite/tests/printer/Ppr020.run Ppr020 [bad exit code] (normal) testsuite/tests/printer/Ppr021.run Ppr021 [bad exit code] (normal) testsuite/tests/printer/Ppr022.run Ppr022 [bad exit code] (normal) testsuite/tests/printer/Ppr023.run Ppr023 [bad exit code] (normal) testsuite/tests/printer/Ppr024.run Ppr024 [bad exit code] (normal) testsuite/tests/printer/Ppr025.run Ppr025 [bad exit code] (normal) testsuite/tests/printer/Ppr026.run Ppr026 [bad exit code] (normal) testsuite/tests/printer/Ppr027.run Ppr027 [bad exit code] (normal) testsuite/tests/printer/Ppr028.run Ppr028 [bad exit code] (normal) testsuite/tests/printer/Ppr029.run Ppr029 [bad exit code] (normal) testsuite/tests/printer/Ppr030.run Ppr030 [bad exit code] (normal) testsuite/tests/printer/Ppr031.run Ppr031 [bad exit code] (normal) testsuite/tests/printer/Ppr032.run Ppr032 [bad exit code] (normal) testsuite/tests/printer/Ppr033.run Ppr033 [bad exit code] (normal) testsuite/tests/printer/Ppr034.run Ppr034 [bad exit code] (normal) testsuite/tests/printer/Ppr035.run Ppr035 [bad exit code] (normal) testsuite/tests/printer/Ppr036.run Ppr036 [bad exit code] (normal) testsuite/tests/printer/Ppr037.run Ppr037 [bad exit code] (normal) testsuite/tests/printer/Ppr038.run Ppr038 [bad exit code] (normal) testsuite/tests/printer/Ppr039.run Ppr039 [bad exit code] (normal) testsuite/tests/printer/Ppr040.run Ppr040 [bad exit code] (normal) testsuite/tests/printer/Ppr041.run Ppr041 [bad exit code] (normal) testsuite/tests/printer/Ppr042.run Ppr042 [bad exit code] (normal) testsuite/tests/printer/Ppr043.run Ppr043 [bad exit code] (normal) testsuite/tests/printer/Ppr044.run Ppr044 [bad exit code] (normal) testsuite/tests/printer/Ppr045.run Ppr045 [bad exit code] (normal) testsuite/tests/printer/Ppr046.run Ppr046 [bad exit code] (normal) testsuite/tests/printer/Ppr048.run Ppr048 [bad exit code] (normal) testsuite/tests/printer/T13199.run T13199 [bad exit code] (normal) testsuite/tests/printer/T13550.run T13550 [bad exit code] (normal) testsuite/tests/printer/T13050p.run T13050p [bad exit code] (normal) testsuite/tests/printer/T13942.run T13942 [bad exit code] (normal) testsuite/tests/printer/T14289.run T14289 [bad exit code] (normal) testsuite/tests/printer/T14289b.run T14289b [bad exit code] (normal) testsuite/tests/printer/T14289c.run T14289c [bad exit code] (normal) testsuite/tests/printer/T14306.run T14306 [bad exit code] (normal) testsuite/tests/profiling/should_compile/T14931.run T14931 [bad exit code] (normal) testsuite/tests/profiling/should_run/T11489.run T11489 [bad exit code] (normal) testsuite/tests/profiling/should_run/scc001.run scc001 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/scc002.run scc002 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/scc003.run scc003 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/scc005.run scc005 [bad profile] (profasm) testsuite/tests/profiling/should_run/scc005.run scc005 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/T680.run T680 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/T2552.run T2552 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/prof-doc-fib.run prof-doc-fib [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/prof-doc-last.run prof-doc-last [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/T5559.run T5559 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/profinline001.run profinline001 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/T11627b.run T11627b [bad exit code] (prof_hc_hb) testsuite/tests/profiling/should_run/T5363.run T5363 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/T14257.run T14257 [bad exit code] (prof) testsuite/tests/profiling/should_run/T14257.run T14257 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/T12962.run T12962 [bad exit code] (ghci-ext-prof) testsuite/tests/quasiquotation/T14028.run T14028 [bad exit code] (normal) testsuite/tests/rename/prog006/rn.prog006.run rn.prog006 [bad exit code] (normal) testsuite/tests/rename/should_compile/T1792_imports.run T1792_imports [bad exit code] (normal) testsuite/tests/rename/should_compile/T3823.run T3823 [bad exit code] (normal) testsuite/tests/rename/should_compile/T4003.run T4003 [bad exit code] (normal) testsuite/tests/rename/should_compile/T3449.run T3449 [bad exit code] (normal) testsuite/tests/rename/should_compile/T4239.run T4239 [bad exit code] (normal) testsuite/tests/rename/should_compile/T4240.run T4240 [bad exit code] (normal) testsuite/tests/rename/should_compile/T7963.run T7963 [bad exit code] (normal) testsuite/tests/rename/should_compile/T7969.run T7969 [bad exit code] (normal) testsuite/tests/rename/should_compile/T3103/T3103.run T3103 [exit code non-0] (normal) testsuite/tests/rename/should_fail/T9032.run T9032 [bad exit code] (normal) testsuite/tests/roles/should_fail/Roles12.run Roles12 [bad exit code] (normal) testsuite/tests/roles/should_fail/T9204.run T9204 [bad exit code] (normal) testsuite/tests/rts/outofmem.run outofmem [bad exit code] (normal) testsuite/tests/rts/outofmem2.run outofmem2 [bad exit code] (normal) testsuite/tests/rts/testwsdeque.run testwsdeque [exit code non-0] (threaded1) testsuite/tests/rts/atomicxchg.run atomicxchg [exit code non-0] (threaded1) testsuite/tests/rts/T4059.run T4059 [bad exit code] (normal) testsuite/tests/rts/T2615.run T2615 [bad stdout] (normal) testsuite/tests/rts/T4850.run T4850 [bad exit code] (normal) testsuite/tests/rts/T5423.run T5423 [bad exit code] (normal) testsuite/tests/rts/T5435_v_asm_a.run T5435_v_asm_a [bad exit code] (normal) testsuite/tests/rts/T5435_v_asm_b.run T5435_v_asm_b [bad exit code] (normal) testsuite/tests/rts/exec_signals.run exec_signals [bad exit code] (normal) testsuite/tests/rts/T5435_v_gcc.run T5435_v_gcc [bad exit code] (normal) testsuite/tests/rts/T5435_dyn_asm.run T5435_dyn_asm [bad exit code] (normal) testsuite/tests/rts/T5435_dyn_gcc.run T5435_dyn_gcc [bad exit code] (normal) testsuite/tests/rts/T6006.run T6006 [exit code non-0] (normal) testsuite/tests/rts/T7037.run T7037 [bad exit code] (normal) testsuite/tests/rts/T7040_ghci.run T7040_ghci [bad stderr] (ghci) testsuite/tests/rts/ffishutdown.run ffishutdown [exit code non-0] (threaded1) testsuite/tests/rts/linker_unload.run linker_unload [bad exit code] (normal) testsuite/tests/rts/T8209.run T8209 [bad stderr] (ghci) testsuite/tests/rts/T8124.run T8124 [exit code non-0] (threaded1) testsuite/tests/rts/T8242.run T8242 [bad stderr] (ghci) testsuite/tests/rts/T9078.run T9078 [exit code non-0] (threaded1) testsuite/tests/rts/linker_error1.run linker_error1 [bad exit code] (normal) testsuite/tests/rts/linker_error2.run linker_error2 [bad exit code] (normal) testsuite/tests/rts/linker_error3.run linker_error3 [bad exit code] (normal) testsuite/tests/rts/T9405.run T9405 [bad exit code] (normal) testsuite/tests/rts/T11788.run T11788 [bad exit code] (normal) testsuite/tests/rts/T10296a.run T10296a [bad exit code] (normal) testsuite/tests/rts/T10728.run T10728 [exit code non-0] (threaded2) testsuite/tests/rts/T10296b.run T10296b [exit code non-0] (threaded2) testsuite/tests/rts/T14695.run T14695 [bad exit code] (normal) testsuite/tests/rts/T14900.run T14900 [exit code non-0] (normal) testsuite/tests/rts/InternalCounters.run InternalCounters [bad exit code] (normal) testsuite/tests/rts/numa001.run numa001 [exit code non-0] (debug_numa) testsuite/tests/rts/T14702.run T14702 [exit code non-0] (threaded1) testsuite/tests/rts/T11223/T11223_simple_link.run T11223_simple_link [bad exit code] (normal) testsuite/tests/rts/T11223/T11223_simple_link_lib.run T11223_simple_link_lib [bad exit code] (normal) testsuite/tests/rts/T11223/T11223_simple_duplicate_lib.run T11223_simple_duplicate_lib [bad exit code] (normal) testsuite/tests/rts/T11223/T11223_simple_unused_duplicate_lib.run T11223_simple_unused_duplicate_lib [bad exit code] (normal) testsuite/tests/rts/nursery-chunks1.run nursery-chunks1 [exit code non-0] (threaded1) testsuite/tests/rts/T11223/T11223_link_order_a_b_succeed.run T11223_link_order_a_b_succeed [bad exit code] (normal) testsuite/tests/rts/T11223/T11223_link_order_b_a_succeed.run T11223_link_order_b_a_succeed [bad exit code] (normal) testsuite/tests/rts/T11223/T11223_link_order_a_b_2_fail.run T11223_link_order_a_b_2_fail [bad exit code] (normal) testsuite/tests/rts/T11223/T11223_link_order_b_a_2_succeed.run T11223_link_order_b_a_2_succeed [bad exit code] (normal) testsuite/tests/rts/T15261/T15261a.run T15261a [bad exit code] (normal) testsuite/tests/rts/T15261/T15261b.run T15261b [bad exit code] (normal) testsuite/tests/rts/T1791/T1791.run T1791 [bad exit code] (normal) testsuite/tests/rts/T8308/T8308.run T8308 [bad exit code] (normal) testsuite/tests/rts/T9579/T9579_stackoverflow_rtsnone.run T9579_stackoverflow_rtsnone [bad exit code] (normal) testsuite/tests/rts/T9579/T9579_stackoverflow_rtssome.run T9579_stackoverflow_rtssome [bad exit code] (normal) testsuite/tests/rts/T9579/T9579_stackoverflow_rtsall.run T9579_stackoverflow_rtsall [bad exit code] (normal) testsuite/tests/rts/T9579/T9579_stackoverflow_rtsall_no_suggestions.run T9579_stackoverflow_rtsall_no_suggestions [bad exit code] (normal) testsuite/tests/rts/T9579/T9579_outofheap_rtsnone.run T9579_outofheap_rtsnone [bad exit code] (normal) testsuite/tests/rts/T9579/T9579_outofheap_rtssome.run T9579_outofheap_rtssome [bad exit code] (normal) testsuite/tests/rts/T9579/T9579_outofheap_rtsall.run T9579_outofheap_rtsall [bad exit code] (normal) testsuite/tests/rts/T9579/T9579_outofheap_rtsall_no_suggestions.run T9579_outofheap_rtsall_no_suggestions [bad exit code] (normal) testsuite/tests/runghc/T7859.run T7859 [bad exit code] (normal) testsuite/tests/runghc/T8601.run T8601 [bad exit code] (normal) testsuite/tests/safeHaskell/check/pkg01/safePkg01.run safePkg01 [bad exit code] (normal) testsuite/tests/safeHaskell/check/pkg01/ImpSafeOnly01.run ImpSafeOnly01 [exit code non-0] (normal) testsuite/tests/safeHaskell/check/pkg01/ImpSafeOnly02.run ImpSafeOnly02 [exit code non-0] (normal) testsuite/tests/safeHaskell/check/pkg01/ImpSafeOnly03.run ImpSafeOnly03 [stderr mismatch] (normal) testsuite/tests/safeHaskell/check/pkg01/ImpSafeOnly04.run ImpSafeOnly04 [exit code non-0] (normal) testsuite/tests/safeHaskell/check/pkg01/ImpSafeOnly06.run ImpSafeOnly06 [exit code non-0] (normal) testsuite/tests/safeHaskell/check/pkg01/ImpSafeOnly05.run ImpSafeOnly05 [stderr mismatch] (normal) testsuite/tests/safeHaskell/check/pkg01/ImpSafeOnly07.run ImpSafeOnly07 [stderr mismatch] (normal) testsuite/tests/safeHaskell/check/pkg01/ImpSafeOnly10.run ImpSafeOnly10 [exit code non-0] (normal) testsuite/tests/safeHaskell/check/pkg01/ImpSafeOnly08.run ImpSafeOnly08 [stderr mismatch] (normal) testsuite/tests/safeHaskell/check/pkg01/ImpSafeOnly09.run ImpSafeOnly09 [stderr mismatch] (normal) testsuite/tests/showIface/Orphans.run Orphans [bad exit code] (normal) testsuite/tests/showIface/DocsInHiFile0.run DocsInHiFile0 [bad exit code] (normal) testsuite/tests/showIface/DocsInHiFile1.run DocsInHiFile1 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T3234.run T3234 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T3717.run T3717 [stderr mismatch] (optasm) testsuite/tests/simplCore/should_compile/T3055.run T3055 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T4908.run T4908 [stderr mismatch] (optasm) testsuite/tests/simplCore/should_compile/T4930.run T4930 [stderr mismatch] (optasm) testsuite/tests/simplCore/should_compile/spec-inline.run spec-inline [stderr mismatch] (optasm) testsuite/tests/simplCore/should_compile/T4306.run T4306 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T4201.run T4201 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T3772.run T3772 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/EvalTest.run EvalTest [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T4903.run T4903 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T4918.run T4918 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T4945.run T4945 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T4957.run T4957 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T5168.run T5168 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/simpl021.run simpl021 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T5327.run T5327 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T5615.run T5615 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T5623.run T5623 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T13155.run T13155 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T5776.run T5776 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T5658b.run T5658b [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T4138.run T4138 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T7165.run T7165 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T5366.run T5366 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T7796.run T7796 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T7360.run T7360 [stderr mismatch] (optasm) testsuite/tests/simplCore/should_compile/T7865.run T7865 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T8221.run T8221 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T8274.run T8274 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T5996.run T5996 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T8832.run T8832 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T8848.run T8848 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T10083.run T10083 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T11155.run T11155 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T3990.run T3990 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T9509.run T9509 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T12603.run T12603 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T12877.run T12877 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T13025.run T13025 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T13156.run T13156 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T13143.run T13143 [stderr mismatch] (optasm) testsuite/tests/simplCore/should_compile/str-rules.run str-rules [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T13317.run T13317 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T13340.run T13340 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T13367.run T13367 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T13468.run T13468 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T11272.run T11272 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T12600.run T12600 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T13543.run T13543 [stderr mismatch] (optasm) testsuite/tests/simplCore/should_compile/T14140.run T14140 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T14978.run T14978 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T15056.run T15056 [stderr mismatch] (normal) testsuite/tests/stranal/should_compile/T10694.run T10694 [stderr mismatch] (optasm) testsuite/tests/stranal/sigs/HyperStrUse.run HyperStrUse [stderr mismatch] (optasm) testsuite/tests/stranal/sigs/T12370.run T12370 [stderr mismatch] (optasm) testsuite/tests/stranal/sigs/CaseBinderCPR.run CaseBinderCPR [stderr mismatch] (optasm) testsuite/tests/th/TH_spliceE5_prof.run TH_spliceE5_prof [bad exit code] (normal) testsuite/tests/th/TH_spliceE5_prof_ext.run TH_spliceE5_prof_ext [bad exit code] (normal) testsuite/tests/th/T2386.run T2386 [bad exit code] (normal) testsuite/tests/th/TH_Depends.run TH_Depends [bad exit code] (normal) testsuite/tests/th/T7445.run T7445 [bad exit code] (normal) testsuite/tests/th/T8333.run T8333 [bad exit code] (normal) testsuite/tests/th/T8624.run T8624 [bad exit code] (normal) testsuite/tests/th/T2014/T2014.run T2014 [bad exit code] (normal) testsuite/tests/typecheck/T13168/T13168.run T13168 [bad exit code] (normal) testsuite/tests/typecheck/bug1465/bug1465.run bug1465 [bad exit code] (normal) testsuite/tests/typecheck/should_compile/tc170.run tc170 [bad exit code] (normal) testsuite/tests/typecheck/should_compile/tc173.run tc173 [bad exit code] (normal) testsuite/tests/typecheck/should_compile/tc245.run tc245 [bad exit code] (normal) testsuite/tests/typecheck/should_compile/T2412.run T2412 [bad exit code] (normal) testsuite/tests/typecheck/should_compile/tc266.run tc266 [bad exit code] (normal) testsuite/tests/typecheck/should_compile/Tc267.run Tc267 [bad exit code] (normal) testsuite/tests/typecheck/should_compile/Tc271.run Tc271 [bad exit code] (normal) testsuite/tests/typecheck/should_compile/T5792.run T5792 [bad exit code] (normal) testsuite/tests/typecheck/should_compile/T7171.run T7171 [bad exit code] (normal) testsuite/tests/typecheck/should_compile/T13585.run T13585 [bad exit code] (normal) testsuite/tests/typecheck/should_compile/T14434.run T14434 [bad exit code] (normal) testsuite/tests/typecheck/should_compile/T14774.run T14774 [bad exit code] (normal) testsuite/tests/typecheck/should_compile/T14934.run T14934 [bad exit code] (normal) testsuite/tests/unboxedsums/module/sum_mod.run sum_mod [bad exit code] (normal) libraries/hpc/tests/ghc_ghci/hpc_ghc_ghci.run hpc_ghc_ghci [bad exit code] (normal) libraries/base/tests/memo001.run memo001 [exit code non-0] (normal) libraries/base/tests/memo002.run memo002 [exit code non-0] (normal) libraries/base/tests/IO/T3307.run T3307 [bad exit code] (normal) libraries/base/tests/IO/environment001.run environment001 [bad exit code] (normal) libraries/base/tests/IO/T12010/T12010.run T12010 [bad exit code] (threaded1) libraries/ghc-compact/tests/compact_simple.run compact_simple [exit code non-0] (normal) libraries/ghc-compact/tests/compact_simple.run compact_simple [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_loop.run compact_loop [exit code non-0] (normal) libraries/ghc-compact/tests/compact_loop.run compact_loop [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_append.run compact_append [exit code non-0] (normal) libraries/ghc-compact/tests/compact_autoexpand.run compact_autoexpand [exit code non-0] (normal) libraries/ghc-compact/tests/compact_append.run compact_append [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_autoexpand.run compact_autoexpand [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_huge_array.run compact_huge_array [exit code non-0] (normal) libraries/ghc-compact/tests/compact_simple_array.run compact_simple_array [exit code non-0] (normal) libraries/ghc-compact/tests/compact_serialize.run compact_serialize [exit code non-0] (normal) libraries/ghc-compact/tests/compact_huge_array.run compact_huge_array [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_simple_array.run compact_simple_array [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_largemap.run compact_largemap [exit code non-0] (normal) libraries/ghc-compact/tests/compact_serialize.run compact_serialize [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_threads.run compact_threads [exit code non-0] (normal) libraries/ghc-compact/tests/compact_largemap.run compact_largemap [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_cycle.run compact_cycle [exit code non-0] (normal) libraries/ghc-compact/tests/compact_threads.run compact_threads [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_function.run compact_function [exit code non-0] (normal) libraries/ghc-compact/tests/compact_cycle.run compact_cycle [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_mutable.run compact_mutable [exit code non-0] (normal) libraries/ghc-compact/tests/compact_function.run compact_function [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_pinned.run compact_pinned [exit code non-0] (normal) libraries/ghc-compact/tests/compact_mutable.run compact_mutable [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_gc.run compact_gc [exit code non-0] (normal) libraries/ghc-compact/tests/compact_pinned.run compact_pinned [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_share.run compact_share [exit code non-0] (normal) libraries/ghc-compact/tests/compact_bench.run compact_bench [exit code non-0] (normal) libraries/ghc-compact/tests/compact_gc.run compact_gc [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_share.run compact_share [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_bench.run compact_bench [exit code non-0] (sanity) libraries/process/tests/process007.run process007 [bad exit code] (normal) libraries/process/tests/T3231.run T3231 [exit code non-0] (threaded1) libraries/process/tests/T3994.run T3994 [exit code non-0] (threaded1) libraries/stm/tests/stm056.run stm056 [exit code non-0] (threaded1) libraries/unix/tests/forkprocess01.run forkprocess01 [exit code non-0] (threaded1_ls) testsuite/tests/rts/T7815.run T7815 [exit code non-0] (threaded1) libraries/base/tests/IO/concio001.run concio001 [bad exit code] (normal) libraries/base/tests/IO/concio001.thr.run concio001.thr [bad exit code] (normal) Unexpected stat failures: testsuite/tests/callarity/perf/T3924.run T3924 [stat not good enough] (normal) testsuite/tests/perf/join_points/join004.run join004 [stat not good enough] (optasm) testsuite/tests/perf/should_run/T10359.run T10359 [stat not good enough] (normal) testsuite/tests/perf/should_run/T4830.run T4830 [stat not good enough] (normal) testsuite/tests/perf/should_run/T14955.run T14955 [stat not good enough] (normal) testsuite/tests/perf/join_points/join007.run join007 [stat not good enough] (optasm) testsuite/tests/perf/should_run/T3586.run T3586 [stat not good enough] (normal) testsuite/tests/perf/should_run/lazy-bs-alloc.run lazy-bs-alloc [stat not good enough] (normal) testsuite/tests/perf/should_run/T876.run T876 [stat not good enough] (normal) testsuite/tests/perf/should_run/T3738.run T3738 [stat not good enough] (normal) testsuite/tests/perf/should_run/T5113.run T5113 [stat not good enough] (normal) testsuite/tests/perf/should_run/MethSharing.run MethSharing [stat not good enough] (normal) testsuite/tests/perf/should_run/T4978.run T4978 [stat not good enough] (normal) testsuite/tests/perf/should_run/T5205.run T5205 [stat not good enough] (normal) testsuite/tests/perf/should_run/T4474c.run T4474c [stat not good enough] (normal) testsuite/tests/perf/should_run/T4474a.run T4474a [stat not good enough] (normal) testsuite/tests/perf/should_run/T4474b.run T4474b [stat not good enough] (normal) testsuite/tests/perf/should_run/T5237.run T5237 [stat not good enough] (normal) testsuite/tests/perf/should_run/Conversions.run Conversions [stat not good enough] (normal) testsuite/tests/perf/should_run/T7257.run T7257 [stat not good enough] (normal) testsuite/tests/perf/should_run/T5536.run T5536 [stat not good enough] (normal) testsuite/tests/perf/should_run/T7797.run T7797 [stat not good enough] (normal) testsuite/tests/perf/should_run/T5949.run T5949 [stat not good enough] (normal) testsuite/tests/perf/should_run/T4267.run T4267 [stat not good enough] (normal) testsuite/tests/perf/should_run/T7619.run T7619 [stat not good enough] (normal) testsuite/tests/perf/should_run/InlineArrayAlloc.run InlineArrayAlloc [stat not good enough] (normal) testsuite/tests/perf/should_run/InlineByteArrayAlloc.run InlineByteArrayAlloc [stat not good enough] (normal) testsuite/tests/perf/should_run/InlineCloneArrayAlloc.run InlineCloneArrayAlloc [stat not good enough] (normal) testsuite/tests/perf/should_run/T9203.run T9203 [stat not good enough] (normal) testsuite/tests/perf/should_run/T8472.run T8472 [stat not good enough] (normal) testsuite/tests/perf/should_run/T12996.run T12996 [stat not good enough] (normal) testsuite/tests/perf/should_run/T13001.run T13001 [stat not good enough] (normal) testsuite/tests/perf/should_run/T12990.run T12990 [stat not good enough] (normal) testsuite/tests/perf/should_run/DeriveNull.run DeriveNull [stat not good enough] (normal) testsuite/tests/perf/should_run/T14936.run T14936 [stat not good enough] (normal) testsuite/tests/perf/should_run/T13623.run T13623 [stat not good enough] (normal) testsuite/tests/perf/should_run/T15226.run T15226 [stat not good enough] (normal) testsuite/tests/perf/should_run/T15226a.run T15226a [stat not good enough] (normal) testsuite/tests/perf/space_leaks/space_leak_001.run space_leak_001 [stat not good enough] (normal) testsuite/tests/perf/space_leaks/T4334.run T4334 [stat not good enough] (normal) testsuite/tests/perf/should_run/T5549.run T5549 [stat not good enough] (normal) libraries/base/tests/T8766.run T8766 [stat not good enough] (normal) libraries/base/tests/T13191.run T13191 [stat not good enough] (normal) Framework failures: testsuite/tests/ffi/should_run/ffi002.run ffi002 [normal] (pre_cmd failed: 2) testsuite/tests/ffi/should_run/ffi018_ghci.run ffi018_ghci [ghci] (pre_cmd failed: 2) testsuite/tests/ffi/should_run/T1288_ghci.run T1288_ghci [ghci] (pre_cmd failed: 2) testsuite/tests/ffi/should_run/T2276_ghci.run T2276_ghci [ghci] (pre_cmd failed: 2) testsuite/tests/ffi/should_run/T5402.run T5402 [normal] (pre_cmd failed: 2) testsuite/tests/ffi/should_run/T5594.run T5594 [normal] (pre_cmd failed: 2) testsuite/tests/ffi/should_run/ffi023.run ffi023 [normal] (pre_cmd failed: 2) testsuite/tests/ghci/linking/dyn/T1407.run T1407 [normal] (pre_cmd failed: 2) testsuite/tests/ghci/linking/dyn/T10458.run T10458 [ghci] (pre_cmd failed: 2) testsuite/tests/ghci/prog014/prog014.run prog014 [ghci] (pre_cmd failed: 2) testsuite/tests/ghci/scripts/ghci056.run ghci056 [ghci] (pre_cmd failed: 2) testsuite/tests/ghci/scripts/T6106.run T6106 [ghci] (pre_cmd failed: 2) testsuite/tests/ghci/should_run/T3171.run T3171 [normal] ([Errno 2] No such file or directory: 'testsuite/tests/ghci/should_run/T3171.run/T3171.run.stderr') testsuite/tests/perf/haddock/haddock.base.run haddock.base [normal] (extra_file does not exist: ../../../../libraries/base/dist-install/haddock.t) testsuite/tests/perf/haddock/haddock.Cabal.run haddock.Cabal [normal] (extra_file does not exist: ../../../../libraries/Cabal/Cabal/dist-install/haddock.t) testsuite/tests/perf/haddock/haddock.compiler.run haddock.compiler [normal] (extra_file does not exist: ../../../../compiler/stage2/haddock.t) testsuite/tests/plugins/plugins01.run plugins01 [normal] (pre_cmd failed: 2) testsuite/tests/plugins/plugins02.run plugins02 [normal] (pre_cmd failed: 2) testsuite/tests/plugins/plugins03.run plugins03 [normal] (pre_cmd failed: 2) testsuite/tests/plugins/plugins07.run plugins07 [normal] (pre_cmd failed: 2) testsuite/tests/plugins/plugins08.run plugins08 [normal] (pre_cmd failed: 2) testsuite/tests/plugins/plugins09.run plugins09 [normal] (pre_cmd failed: 2) testsuite/tests/plugins/plugins10.run plugins10 [normal] (pre_cmd failed: 2) testsuite/tests/plugins/plugins11.run plugins11 [normal] (pre_cmd failed: 2) testsuite/tests/plugins/plugins12.run plugins12 [normal] (pre_cmd failed: 2) testsuite/tests/plugins/plugins13.run plugins13 [normal] (pre_cmd failed: 2) testsuite/tests/plugins/plugins14.run plugins14 [normal] (pre_cmd failed: 2) testsuite/tests/plugins/plugins15.run plugins15 [normal] (pre_cmd failed: 2) testsuite/tests/plugins/T10420.run T10420 [normal] (pre_cmd failed: 2) testsuite/tests/plugins/T10294.run T10294 [normal] (pre_cmd failed: 2) testsuite/tests/plugins/T10294a.run T10294a [normal] (pre_cmd failed: 2) testsuite/tests/plugins/T11244.run T11244 [normal] (pre_cmd failed: 2) testsuite/tests/plugins/T12567a.run T12567a [normal] (pre_cmd failed: 2) testsuite/tests/plugins/T14335.run T14335 [normal] (pre_cmd failed: 2) testsuite/tests/plugins/plugin-recomp-pure.run plugin-recomp-pure [normal] (pre_cmd failed: 2) testsuite/tests/plugins/plugin-recomp-flags.run plugin-recomp-flags [normal] (pre_cmd failed: 2) testsuite/tests/plugins/plugin-recomp-impure.run plugin-recomp-impure [normal] (pre_cmd failed: 2) testsuite/tests/plugins/plugin-recomp-change.run plugin-recomp-change [normal] (pre_cmd failed: 2) testsuite/tests/plugins/plugin-recomp-change-prof.run plugin-recomp-change-prof [normal] (pre_cmd failed: 2) testsuite/tests/rts/T2615.run T2615 [normal] (pre_cmd failed: 2) testsuite/tests/rts/exec_signals.run exec_signals [normal] (pre_cmd failed: 2) testsuite/tests/rts/T6006.run T6006 [normal] (pre_cmd failed: 2) testsuite/tests/rts/T7040_ghci.run T7040_ghci [ghci] (pre_cmd failed: 2) testsuite/tests/rts/T8124.run T8124 [threaded1] (pre_cmd failed: 2) testsuite/tests/safeHaskell/check/pkg01/ImpSafeOnly01.run ImpSafeOnly01 [normal] (pre_cmd failed: 2) testsuite/tests/safeHaskell/check/pkg01/ImpSafeOnly02.run ImpSafeOnly02 [normal] (pre_cmd failed: 2) testsuite/tests/safeHaskell/check/pkg01/ImpSafeOnly03.run ImpSafeOnly03 [normal] (pre_cmd failed: 2) testsuite/tests/safeHaskell/check/pkg01/ImpSafeOnly05.run ImpSafeOnly05 [normal] (pre_cmd failed: 2) testsuite/tests/safeHaskell/check/pkg01/ImpSafeOnly04.run ImpSafeOnly04 [normal] (pre_cmd failed: 2) testsuite/tests/safeHaskell/check/pkg01/ImpSafeOnly06.run ImpSafeOnly06 [normal] (pre_cmd failed: 2) testsuite/tests/safeHaskell/check/pkg01/ImpSafeOnly07.run ImpSafeOnly07 [normal] (pre_cmd failed: 2) testsuite/tests/safeHaskell/check/pkg01/ImpSafeOnly08.run ImpSafeOnly08 [normal] (pre_cmd failed: 2) testsuite/tests/safeHaskell/check/pkg01/ImpSafeOnly09.run ImpSafeOnly09 [normal] (pre_cmd failed: 2) testsuite/tests/safeHaskell/check/pkg01/ImpSafeOnly10.run ImpSafeOnly10 [normal] (pre_cmd failed: 2) libraries/process/tests/process007.run process007 [normal] (pre_cmd failed: 2) libraries/process/tests/T3994.run T3994 [threaded1] (pre_cmd failed: 2) ```

Let's get this unexpected failures number down. If it's alright, I will use this issue to report the problems that cause those failures and track PRs that address them, along with the new test results that result from my patches. It just didn't feel appropriate to steal existing issues for this.

snowleopard commented 6 years ago

Many thanks @alpmestan!

I will check if there are any outdated testsuite related issues and will close them.

There is also this PR #639, which now seems abandoned. Feel free to close it.

alpmestan commented 6 years ago

hsc2hs is expected under _build/stage1/bin/hsc2hs but not needed by the test rules. This is fixed by https://github.com/snowleopard/hadrian/pull/672.

With this patch, f I only run the failing tests from the first comment, I get:

Test results ``` Unexpected results from: TEST="CaseBinderCPR Conversions DeriveNull HyperStrUse InlineArrayAlloc InlineByteArrayAlloc InlineCloneArrayAlloc InternalCounters MethSharing Ppr001 Ppr002 Ppr003 Ppr004 Ppr005 Ppr006 Ppr007 Ppr008 Ppr009 Ppr010 Ppr011 Ppr012 Ppr013 Ppr014 Ppr015 Ppr016 Ppr017 Ppr018 Ppr019 Ppr020 Ppr021 Ppr022 Ppr023 Ppr024 Ppr025 Ppr026 Ppr027 Ppr028 Ppr029 Ppr030 Ppr031 Ppr032 Ppr033 Ppr034 Ppr035 Ppr036 Ppr037 Ppr038 Ppr039 Ppr040 Ppr041 Ppr042 Ppr043 Ppr044 Ppr045 Ppr046 Ppr048 T10255 T10268 T10269 T10276 T10278 T10280 T10294 T10294a T10296b T10307 T10309 T10312 T10354 T10357 T10358 T10359 T10396 T10399 T10414 T10420 T10598 T10694 T10728 T11018 T11321 T11332 T11627b T12010 T12370 T12417 T12567a T12603 T12962 T12990 T12996 T13001 T13050p T13143 T13163 T13168 T13191 T13199 T13340 T13543 T13550 T13623 T13702 T13942 T14289 T14289b T14289c T14306 T14346 T14465 T14702 T14931 T14936 T14955 T15056 T15226 T15226a T15303 T2552 T2740 T2976 T3055 T3231 T3429 T3586 T3717 T3738 T3772 T3924 T3994 T4267 T4334 T4474a T4474b T4474c T4830 T4850 T4908 T4930 T4978 T5113 T5205 T5237 T5327 T5363 T5536 T5549 T5558 T5559 T5949 T680 T7257 T7360 T7619 T7797 T7815 T7865 T8124 T8209 T8242 T8472 T876 T8766 T8832 T9078 T9203 T9208 T9293 TH_spliceE5_prof TH_spliceE5_prof_ext atomicxchg boolFormula break001 break005 break006 break026 break027 bundle-export conc069 conc069a conc070 conc072 debug dynbrk002 exampleTest ffi014 ffishutdown forkprocess01 ghci055 ghci057 haddock.Cabal haddock.base haddock.compiler hist001 hist002 inline-check integerConstantFolding join004 join007 lazy-bs-alloc linker_error1 linker_error2 linker_error3 load-main memo001 numa001 numsparks001 nursery-chunks1 plugin-recomp-change plugin-recomp-change-prof plugin-recomp-flags plugin-recomp-impure plugin-recomp-pure plugins01 plugins07 plugins09 plugins11 plugins12 plugins13 plugins14 plugins15 print001 print002 print003 print004 print005 print006 print008 print010 print012 print013 print014 print016 print017 print023 print024 print028 prof-doc-fib prof-doc-last profinline001 scc001 scc002 scc003 scc005 space_leak_001 spec-inline stm056 testwsdeque" SUMMARY for test run started at Mon Sep 3 15:31:26 2018 CEST 0:04:11 spent to go through 708 total tests, which gave rise to 1660 test cases, of which 878 were skipped 0 had missing libraries 506 expected passes 2 expected failures 3 caused framework failures 0 caused framework warnings 1 unexpected passes 230 unexpected failures 43 unexpected stat failures Unexpected passes: testsuite/tests/stranal/should_compile/T9208.run T9208 [unexpected] (optasm) Unexpected failures: testsuite/tests/codeGen/should_run/T10414.run T10414 [exit code non-0] (threaded2) testsuite/tests/concurrent/should_run/conc069.run conc069 [bad exit code] (ghci) testsuite/tests/codeGen/should_run/T14346.run T14346 [exit code non-0] (threaded1) testsuite/tests/concurrent/should_run/conc069a.run conc069a [bad exit code] (ghci) testsuite/tests/codeGen/should_compile/debug.run debug [bad stdout] (normal) testsuite/tests/concurrent/should_run/conc070.run conc070 [bad stdout] (ghci) testsuite/tests/concurrent/should_run/conc072.run conc072 [exit code non-0] (threaded2) testsuite/tests/concurrent/should_run/numsparks001.run numsparks001 [exit code non-0] (threaded1) testsuite/tests/concurrent/should_run/T5558.run T5558 [exit code non-0] (threaded2) testsuite/tests/concurrent/should_run/T3429.run T3429 [exit code non-0] (debug_numa) testsuite/tests/driver/inline-check.run inline-check [stderr mismatch] (normal) testsuite/tests/dynlibs/T13702.run T13702 [bad exit code] (normal) testsuite/tests/ffi/should_run/ffi014.run ffi014 [exit code non-0] (threaded1) testsuite/tests/ghc-api/annotations/exampleTest.run exampleTest [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10255.run T10255 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10268.run T10268 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10269.run T10269 [bad stdout] (normal) testsuite/tests/ghc-api/annotations/T10280.run T10280 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10312.run T10312 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10307.run T10307 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10309.run T10309 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/boolFormula.run boolFormula [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10357.run T10357 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10358.run T10358 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10278.run T10278 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10354.run T10354 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10396.run T10396 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10399.run T10399 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T11018.run T11018 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/bundle-export.run bundle-export [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10276.run T10276 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T10598.run T10598 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T11321.run T11321 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T11332.run T11332 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/load-main.run load-main [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T12417.run T12417 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T13163.run T13163 [bad exit code] (normal) testsuite/tests/ghc-api/annotations/T15303.run T15303 [bad exit code] (normal) testsuite/tests/ghci/scripts/ghci055.run ghci055 [bad stdout] (ghci) testsuite/tests/ghci/scripts/ghci057.run ghci057 [bad stdout] (ghci) testsuite/tests/ghci/scripts/T2976.run T2976 [bad stdout] (ghci) testsuite/tests/ghci/scripts/T9293.run T9293 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print001.run print001 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print001.run print001 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print002.run print002 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print003.run print003 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print002.run print002 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print003.run print003 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print004.run print004 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print005.run print005 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print006.run print006 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print005.run print005 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print004.run print004 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print008.run print008 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print006.run print006 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print010.run print010 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print012.run print012 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print013.run print013 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print008.run print008 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print010.run print010 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print012.run print012 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print013.run print013 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print014.run print014 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print016.run print016 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print017.run print017 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print014.run print014 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print023.run print023 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print017.run print017 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print016.run print016 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print024.run print024 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print023.run print023 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print028.run print028 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/break001.run break001 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print024.run print024 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/break005.run break005 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print028.run print028 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/break001.run break001 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/break006.run break006 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/break026.run break026 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/break005.run break005 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/break027.run break027 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/break006.run break006 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/break026.run break026 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/dynbrk002.run dynbrk002 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/break027.run break027 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/hist001.run hist001 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/hist002.run hist002 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/dynbrk002.run dynbrk002 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/T2740.run T2740 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/hist001.run hist001 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/hist002.run hist002 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/T2740.run T2740 [bad stdout] (ghci-ext) testsuite/tests/lib/integer/integerConstantFolding.run integerConstantFolding [bad stdout] (normal) testsuite/tests/numeric/should_compile/T14465.run T14465 [bad stdout] (normal) testsuite/tests/perf/haddock/haddock.base.run haddock.base [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.base.run/haddock.t'] (normal) testsuite/tests/perf/haddock/haddock.Cabal.run haddock.Cabal [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.Cabal.run/haddock.t'] (normal) testsuite/tests/perf/haddock/haddock.compiler.run haddock.compiler [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.compiler.run/haddock.t'] (normal) testsuite/tests/plugins/plugins01.run plugins01 [bad exit code] (normal) testsuite/tests/plugins/plugins07.run plugins07 [bad exit code] (normal) testsuite/tests/plugins/plugins09.run plugins09 [bad exit code] (normal) testsuite/tests/plugins/plugins11.run plugins11 [bad exit code] (normal) testsuite/tests/plugins/plugins12.run plugins12 [bad exit code] (normal) testsuite/tests/plugins/plugins13.run plugins13 [bad exit code] (normal) testsuite/tests/plugins/plugins14.run plugins14 [bad exit code] (normal) testsuite/tests/plugins/plugins15.run plugins15 [bad exit code] (normal) testsuite/tests/plugins/T10420.run T10420 [bad exit code] (normal) testsuite/tests/plugins/T10294.run T10294 [bad exit code] (normal) testsuite/tests/plugins/T10294a.run T10294a [bad exit code] (normal) testsuite/tests/plugins/plugin-recomp-impure.run plugin-recomp-impure [bad exit code] (normal) testsuite/tests/plugins/plugin-recomp-pure.run plugin-recomp-pure [bad exit code] (normal) testsuite/tests/plugins/T12567a.run T12567a [bad exit code] (normal) testsuite/tests/printer/Ppr001.run Ppr001 [bad exit code] (normal) testsuite/tests/printer/Ppr002.run Ppr002 [bad exit code] (normal) testsuite/tests/printer/Ppr003.run Ppr003 [bad exit code] (normal) testsuite/tests/printer/Ppr004.run Ppr004 [bad exit code] (normal) testsuite/tests/plugins/plugin-recomp-flags.run plugin-recomp-flags [bad exit code] (normal) testsuite/tests/plugins/plugin-recomp-change.run plugin-recomp-change [bad exit code] (normal) testsuite/tests/printer/Ppr005.run Ppr005 [bad exit code] (normal) testsuite/tests/printer/Ppr006.run Ppr006 [bad exit code] (normal) testsuite/tests/printer/Ppr007.run Ppr007 [bad exit code] (normal) testsuite/tests/printer/Ppr008.run Ppr008 [bad exit code] (normal) testsuite/tests/printer/Ppr009.run Ppr009 [bad exit code] (normal) testsuite/tests/plugins/plugin-recomp-change-prof.run plugin-recomp-change-prof [bad exit code] (normal) testsuite/tests/printer/Ppr010.run Ppr010 [bad exit code] (normal) testsuite/tests/printer/Ppr011.run Ppr011 [bad exit code] (normal) testsuite/tests/printer/Ppr013.run Ppr013 [bad exit code] (normal) testsuite/tests/printer/Ppr012.run Ppr012 [bad exit code] (normal) testsuite/tests/printer/Ppr014.run Ppr014 [bad exit code] (normal) testsuite/tests/printer/Ppr015.run Ppr015 [bad exit code] (normal) testsuite/tests/printer/Ppr017.run Ppr017 [bad exit code] (normal) testsuite/tests/printer/Ppr016.run Ppr016 [bad exit code] (normal) testsuite/tests/printer/Ppr018.run Ppr018 [bad exit code] (normal) testsuite/tests/printer/Ppr019.run Ppr019 [bad exit code] (normal) testsuite/tests/printer/Ppr020.run Ppr020 [bad exit code] (normal) testsuite/tests/printer/Ppr021.run Ppr021 [bad exit code] (normal) testsuite/tests/printer/Ppr022.run Ppr022 [bad exit code] (normal) testsuite/tests/printer/Ppr023.run Ppr023 [bad exit code] (normal) testsuite/tests/printer/Ppr024.run Ppr024 [bad exit code] (normal) testsuite/tests/printer/Ppr025.run Ppr025 [bad exit code] (normal) testsuite/tests/printer/Ppr026.run Ppr026 [bad exit code] (normal) testsuite/tests/printer/Ppr027.run Ppr027 [bad exit code] (normal) testsuite/tests/printer/Ppr028.run Ppr028 [bad exit code] (normal) testsuite/tests/printer/Ppr029.run Ppr029 [bad exit code] (normal) testsuite/tests/printer/Ppr030.run Ppr030 [bad exit code] (normal) testsuite/tests/printer/Ppr031.run Ppr031 [bad exit code] (normal) testsuite/tests/printer/Ppr033.run Ppr033 [bad exit code] (normal) testsuite/tests/printer/Ppr032.run Ppr032 [bad exit code] (normal) testsuite/tests/printer/Ppr034.run Ppr034 [bad exit code] (normal) testsuite/tests/printer/Ppr035.run Ppr035 [bad exit code] (normal) testsuite/tests/printer/Ppr036.run Ppr036 [bad exit code] (normal) testsuite/tests/printer/Ppr037.run Ppr037 [bad exit code] (normal) testsuite/tests/printer/Ppr038.run Ppr038 [bad exit code] (normal) testsuite/tests/printer/Ppr039.run Ppr039 [bad exit code] (normal) testsuite/tests/printer/Ppr041.run Ppr041 [bad exit code] (normal) testsuite/tests/printer/Ppr040.run Ppr040 [bad exit code] (normal) testsuite/tests/printer/Ppr042.run Ppr042 [bad exit code] (normal) testsuite/tests/printer/Ppr043.run Ppr043 [bad exit code] (normal) testsuite/tests/printer/Ppr045.run Ppr045 [bad exit code] (normal) testsuite/tests/printer/Ppr044.run Ppr044 [bad exit code] (normal) testsuite/tests/printer/Ppr046.run Ppr046 [bad exit code] (normal) testsuite/tests/printer/Ppr048.run Ppr048 [bad exit code] (normal) testsuite/tests/printer/T13199.run T13199 [bad exit code] (normal) testsuite/tests/printer/T13050p.run T13050p [bad exit code] (normal) testsuite/tests/printer/T13550.run T13550 [bad exit code] (normal) testsuite/tests/printer/T13942.run T13942 [bad exit code] (normal) testsuite/tests/printer/T14289.run T14289 [bad exit code] (normal) testsuite/tests/printer/T14289b.run T14289b [bad exit code] (normal) testsuite/tests/printer/T14289c.run T14289c [bad exit code] (normal) testsuite/tests/printer/T14306.run T14306 [bad exit code] (normal) testsuite/tests/profiling/should_compile/T14931.run T14931 [bad exit code] (normal) testsuite/tests/profiling/should_run/scc001.run scc001 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/scc005.run scc005 [bad profile] (profasm) testsuite/tests/profiling/should_run/scc002.run scc002 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/scc003.run scc003 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/scc005.run scc005 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/T680.run T680 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/prof-doc-fib.run prof-doc-fib [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/T2552.run T2552 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/T5559.run T5559 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/prof-doc-last.run prof-doc-last [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/profinline001.run profinline001 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/T12962.run T12962 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/T11627b.run T11627b [bad exit code] (prof_hc_hb) testsuite/tests/profiling/should_run/T5363.run T5363 [bad exit code] (ghci-ext-prof) testsuite/tests/rts/atomicxchg.run atomicxchg [exit code non-0] (threaded1) testsuite/tests/rts/testwsdeque.run testwsdeque [exit code non-0] (threaded1) testsuite/tests/rts/T4850.run T4850 [bad exit code] (normal) testsuite/tests/rts/ffishutdown.run ffishutdown [exit code non-0] (threaded1) testsuite/tests/rts/T8209.run T8209 [bad stderr] (ghci) testsuite/tests/rts/T8242.run T8242 [bad stderr] (ghci) testsuite/tests/rts/T9078.run T9078 [exit code non-0] (threaded1) testsuite/tests/rts/linker_error1.run linker_error1 [bad exit code] (normal) testsuite/tests/rts/T8124.run T8124 [exit code non-0] (threaded1) testsuite/tests/rts/linker_error2.run linker_error2 [bad exit code] (normal) testsuite/tests/rts/linker_error3.run linker_error3 [bad exit code] (normal) testsuite/tests/rts/T10728.run T10728 [exit code non-0] (threaded2) testsuite/tests/rts/T10296b.run T10296b [exit code non-0] (threaded2) testsuite/tests/rts/numa001.run numa001 [exit code non-0] (debug_numa) testsuite/tests/rts/T14702.run T14702 [exit code non-0] (threaded1) testsuite/tests/rts/InternalCounters.run InternalCounters [bad stdout] (normal) testsuite/tests/rts/nursery-chunks1.run nursery-chunks1 [exit code non-0] (threaded1) testsuite/tests/simplCore/should_compile/T3717.run T3717 [stderr mismatch] (optasm) testsuite/tests/simplCore/should_compile/spec-inline.run spec-inline [stderr mismatch] (optasm) testsuite/tests/simplCore/should_compile/T4908.run T4908 [stderr mismatch] (optasm) testsuite/tests/simplCore/should_compile/T4930.run T4930 [stderr mismatch] (optasm) testsuite/tests/simplCore/should_compile/T3055.run T3055 [bad stdout] (normal) testsuite/tests/simplCore/should_compile/T3772.run T3772 [bad stdout] (normal) testsuite/tests/simplCore/should_compile/T5327.run T5327 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T7360.run T7360 [stderr mismatch] (optasm) testsuite/tests/simplCore/should_compile/T7865.run T7865 [bad stdout] (normal) testsuite/tests/simplCore/should_compile/T8832.run T8832 [bad stdout] (normal) testsuite/tests/simplCore/should_compile/T12603.run T12603 [bad stdout] (normal) testsuite/tests/simplCore/should_compile/T13143.run T13143 [stderr mismatch] (optasm) testsuite/tests/simplCore/should_compile/T13340.run T13340 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T13543.run T13543 [stderr mismatch] (optasm) testsuite/tests/simplCore/should_compile/T15056.run T15056 [stderr mismatch] (normal) testsuite/tests/stranal/should_compile/T10694.run T10694 [stderr mismatch] (optasm) testsuite/tests/stranal/sigs/HyperStrUse.run HyperStrUse [stderr mismatch] (optasm) testsuite/tests/stranal/sigs/T12370.run T12370 [stderr mismatch] (optasm) testsuite/tests/stranal/sigs/CaseBinderCPR.run CaseBinderCPR [stderr mismatch] (optasm) testsuite/tests/th/TH_spliceE5_prof_ext.run TH_spliceE5_prof_ext [bad exit code] (normal) testsuite/tests/th/TH_spliceE5_prof.run TH_spliceE5_prof [bad exit code] (normal) testsuite/tests/typecheck/T13168/T13168.run T13168 [bad stderr] (normal) libraries/base/tests/memo001.run memo001 [bad exit code] (normal) libraries/base/tests/IO/T12010/T12010.run T12010 [bad exit code] (threaded1) libraries/process/tests/T3231.run T3231 [exit code non-0] (threaded1) libraries/stm/tests/stm056.run stm056 [exit code non-0] (threaded1) libraries/unix/tests/forkprocess01.run forkprocess01 [exit code non-0] (threaded1_ls) libraries/process/tests/T3994.run T3994 [exit code non-0] (threaded1) testsuite/tests/rts/T7815.run T7815 [exit code non-0] (threaded1) Unexpected stat failures: testsuite/tests/callarity/perf/T3924.run T3924 [stat not good enough] (normal) testsuite/tests/perf/join_points/join004.run join004 [stat not good enough] (optasm) testsuite/tests/perf/should_run/T10359.run T10359 [stat not good enough] (normal) testsuite/tests/perf/should_run/T14955.run T14955 [stat not good enough] (normal) testsuite/tests/perf/should_run/T3586.run T3586 [stat not good enough] (normal) testsuite/tests/perf/join_points/join007.run join007 [stat not good enough] (optasm) testsuite/tests/perf/should_run/T4830.run T4830 [stat not good enough] (normal) testsuite/tests/perf/should_run/lazy-bs-alloc.run lazy-bs-alloc [stat not good enough] (normal) testsuite/tests/perf/should_run/T3738.run T3738 [stat not good enough] (normal) testsuite/tests/perf/should_run/T876.run T876 [stat not good enough] (normal) testsuite/tests/perf/should_run/MethSharing.run MethSharing [stat not good enough] (normal) testsuite/tests/perf/should_run/T5113.run T5113 [stat not good enough] (normal) testsuite/tests/perf/should_run/T4978.run T4978 [stat not good enough] (normal) testsuite/tests/perf/should_run/T5205.run T5205 [stat not good enough] (normal) testsuite/tests/perf/should_run/T4474a.run T4474a [stat not good enough] (normal) testsuite/tests/perf/should_run/T4474b.run T4474b [stat not good enough] (normal) testsuite/tests/perf/should_run/T4474c.run T4474c [stat not good enough] (normal) testsuite/tests/perf/should_run/T5237.run T5237 [stat not good enough] (normal) testsuite/tests/perf/should_run/T7257.run T7257 [stat not good enough] (normal) testsuite/tests/perf/should_run/Conversions.run Conversions [stat not good enough] (normal) testsuite/tests/perf/should_run/T5536.run T5536 [stat not good enough] (normal) testsuite/tests/perf/should_run/T7797.run T7797 [stat not good enough] (normal) testsuite/tests/perf/should_run/T5949.run T5949 [stat not good enough] (normal) testsuite/tests/perf/should_run/T4267.run T4267 [stat not good enough] (normal) testsuite/tests/perf/should_run/T7619.run T7619 [stat not good enough] (normal) testsuite/tests/perf/should_run/InlineByteArrayAlloc.run InlineByteArrayAlloc [stat not good enough] (normal) testsuite/tests/perf/should_run/InlineArrayAlloc.run InlineArrayAlloc [stat not good enough] (normal) testsuite/tests/perf/should_run/InlineCloneArrayAlloc.run InlineCloneArrayAlloc [stat not good enough] (normal) testsuite/tests/perf/should_run/T8472.run T8472 [stat not good enough] (normal) testsuite/tests/perf/should_run/T9203.run T9203 [stat not good enough] (normal) testsuite/tests/perf/should_run/T12996.run T12996 [stat not good enough] (normal) testsuite/tests/perf/should_run/T13001.run T13001 [stat not good enough] (normal) testsuite/tests/perf/should_run/T12990.run T12990 [stat not good enough] (normal) testsuite/tests/perf/should_run/DeriveNull.run DeriveNull [stat not good enough] (normal) testsuite/tests/perf/should_run/T14936.run T14936 [stat not good enough] (normal) testsuite/tests/perf/should_run/T13623.run T13623 [stat not good enough] (normal) testsuite/tests/perf/should_run/T15226.run T15226 [stat not good enough] (normal) testsuite/tests/perf/should_run/T15226a.run T15226a [stat not good enough] (normal) testsuite/tests/perf/space_leaks/space_leak_001.run space_leak_001 [stat not good enough] (normal) testsuite/tests/perf/space_leaks/T4334.run T4334 [stat not good enough] (normal) testsuite/tests/perf/should_run/T5549.run T5549 [stat not good enough] (normal) libraries/base/tests/T8766.run T8766 [stat not good enough] (normal) libraries/base/tests/T13191.run T13191 [stat not good enough] (normal) ```

So, 500 unexpected failures are gone with the PR.

Now, as a note for future me, I've noticed that check-api-annotations is expected under inplace/.... This is because of some code in testsuite/mk/boilerplate.mk which just doesn't leave any other option at the moment. I guess I'll have to modify it to allow hadrian to completely bypass this and specify a path of its own with a new dedicated variable.

alpmestan commented 6 years ago

https://phabricator.haskell.org/D5127 should fix all the ghc-api-annotations failures. We do set the CHECK_API_ANNOTATIONS environment variable but it is not picked up. testsuite/mk/boilerplate.mk, which is included by many (most? all?) test makefiles, didn't give us a chance to specify anything.

alpmestan commented 6 years ago

With the aforementionned patch, I'm down to 184 unexpected failures

Test results ``` SUMMARY for test run started at Tue Sep 4 18:18:34 2018 CEST 0:19:48 spent to go through 6536 total tests, which gave rise to 24042 test cases, of which 17523 were skipped 27 had missing libraries 6168 expected passes 96 expected failures 3 caused framework failures 0 caused framework warnings 1 unexpected passes 184 unexpected failures 43 unexpected stat failures Unexpected passes: testsuite/tests/stranal/should_compile/T9208.run T9208 [unexpected] (optasm) Unexpected failures: testsuite/tests/codeGen/should_compile/debug.run debug [bad stdout] (normal) testsuite/tests/codeGen/should_run/T10414.run T10414 [exit code non-0] (threaded2) testsuite/tests/codeGen/should_run/T14346.run T14346 [exit code non-0] (threaded1) testsuite/tests/concurrent/should_run/conc069.run conc069 [bad exit code] (ghci) testsuite/tests/concurrent/should_run/conc069a.run conc069a [bad exit code] (ghci) testsuite/tests/concurrent/should_run/conc070.run conc070 [bad stdout] (ghci) testsuite/tests/concurrent/should_run/conc072.run conc072 [exit code non-0] (threaded2) testsuite/tests/concurrent/should_run/numsparks001.run numsparks001 [exit code non-0] (threaded1) testsuite/tests/concurrent/should_run/T3429.run T3429 [exit code non-0] (debug_numa) testsuite/tests/concurrent/should_run/T5558.run T5558 [exit code non-0] (threaded2) testsuite/tests/driver/inline-check.run inline-check [stderr mismatch] (normal) testsuite/tests/dynlibs/T13702.run T13702 [bad exit code] (normal) testsuite/tests/ffi/should_run/ffi014.run ffi014 [exit code non-0] (threaded1) testsuite/tests/ghci/scripts/ghci055.run ghci055 [bad stdout] (ghci) testsuite/tests/ghci/scripts/ghci057.run ghci057 [bad stdout] (ghci) testsuite/tests/ghci/scripts/T2976.run T2976 [bad stdout] (ghci) testsuite/tests/ghci/scripts/T9293.run T9293 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print001.run print001 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print002.run print002 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print001.run print001 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print003.run print003 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print002.run print002 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print004.run print004 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print005.run print005 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print003.run print003 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print004.run print004 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print005.run print005 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print006.run print006 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print008.run print008 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print006.run print006 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print008.run print008 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print010.run print010 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print012.run print012 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print010.run print010 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print012.run print012 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print013.run print013 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print014.run print014 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print016.run print016 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print013.run print013 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print014.run print014 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print016.run print016 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print017.run print017 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print017.run print017 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print023.run print023 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print024.run print024 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print023.run print023 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print024.run print024 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/print028.run print028 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/print028.run print028 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/break001.run break001 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/break001.run break001 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/break005.run break005 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/break006.run break006 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/break005.run break005 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/break006.run break006 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/break026.run break026 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/break027.run break027 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/break026.run break026 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/break027.run break027 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/dynbrk002.run dynbrk002 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/dynbrk002.run dynbrk002 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/hist001.run hist001 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/hist002.run hist002 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/T2740.run T2740 [bad stdout] (ghci) testsuite/tests/ghci.debugger/scripts/hist001.run hist001 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/hist002.run hist002 [bad stdout] (ghci-ext) testsuite/tests/ghci.debugger/scripts/T2740.run T2740 [bad stdout] (ghci-ext) testsuite/tests/lib/integer/integerConstantFolding.run integerConstantFolding [bad stdout] (normal) testsuite/tests/numeric/should_compile/T14465.run T14465 [bad stdout] (normal) testsuite/tests/perf/haddock/haddock.base.run haddock.base [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.base.run/haddock.t'] (normal) testsuite/tests/perf/haddock/haddock.Cabal.run haddock.Cabal [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.Cabal.run/haddock.t'] (normal) testsuite/tests/perf/haddock/haddock.compiler.run haddock.compiler [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.compiler.run/haddock.t'] (normal) testsuite/tests/plugins/plugins01.run plugins01 [bad exit code] (normal) testsuite/tests/plugins/plugins07.run plugins07 [bad exit code] (normal) testsuite/tests/plugins/plugins09.run plugins09 [bad exit code] (normal) testsuite/tests/plugins/plugins11.run plugins11 [bad exit code] (normal) testsuite/tests/plugins/plugins12.run plugins12 [bad exit code] (normal) testsuite/tests/plugins/plugins13.run plugins13 [bad exit code] (normal) testsuite/tests/plugins/T10420.run T10420 [bad exit code] (normal) testsuite/tests/plugins/plugins14.run plugins14 [bad exit code] (normal) testsuite/tests/plugins/plugins15.run plugins15 [bad exit code] (normal) testsuite/tests/plugins/T10294.run T10294 [bad exit code] (normal) testsuite/tests/plugins/T10294a.run T10294a [bad exit code] (normal) testsuite/tests/plugins/plugin-recomp-pure.run plugin-recomp-pure [bad exit code] (normal) testsuite/tests/plugins/T12567a.run T12567a [bad exit code] (normal) testsuite/tests/plugins/plugin-recomp-impure.run plugin-recomp-impure [bad exit code] (normal) testsuite/tests/plugins/plugin-recomp-flags.run plugin-recomp-flags [bad exit code] (normal) testsuite/tests/plugins/plugin-recomp-change.run plugin-recomp-change [bad exit code] (normal) testsuite/tests/plugins/plugin-recomp-change-prof.run plugin-recomp-change-prof [bad exit code] (normal) testsuite/tests/profiling/should_compile/T14931.run T14931 [bad exit code] (normal) testsuite/tests/profiling/should_run/scc001.run scc001 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/scc002.run scc002 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/scc003.run scc003 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/scc005.run scc005 [bad profile] (profasm) testsuite/tests/profiling/should_run/scc005.run scc005 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/T680.run T680 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/T2552.run T2552 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/prof-doc-fib.run prof-doc-fib [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/prof-doc-last.run prof-doc-last [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/T5559.run T5559 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/profinline001.run profinline001 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/T5363.run T5363 [bad exit code] (ghci-ext-prof) testsuite/tests/profiling/should_run/T11627b.run T11627b [bad exit code] (prof_hc_hb) testsuite/tests/profiling/should_run/T12962.run T12962 [bad exit code] (ghci-ext-prof) testsuite/tests/rts/testwsdeque.run testwsdeque [exit code non-0] (threaded1) testsuite/tests/rts/atomicxchg.run atomicxchg [exit code non-0] (threaded1) testsuite/tests/rts/T4850.run T4850 [bad exit code] (normal) testsuite/tests/rts/ffishutdown.run ffishutdown [exit code non-0] (threaded1) testsuite/tests/rts/T8209.run T8209 [bad stderr] (ghci) testsuite/tests/rts/T8242.run T8242 [bad stderr] (ghci) testsuite/tests/rts/T8124.run T8124 [exit code non-0] (threaded1) testsuite/tests/rts/T9078.run T9078 [exit code non-0] (threaded1) testsuite/tests/rts/linker_error1.run linker_error1 [bad exit code] (normal) testsuite/tests/rts/linker_error2.run linker_error2 [bad exit code] (normal) testsuite/tests/rts/linker_error3.run linker_error3 [bad exit code] (normal) testsuite/tests/rts/T10728.run T10728 [exit code non-0] (threaded2) testsuite/tests/rts/T10296b.run T10296b [exit code non-0] (threaded2) testsuite/tests/rts/numa001.run numa001 [exit code non-0] (debug_numa) testsuite/tests/rts/T14900.run T14900 [exit code non-0] (normal) testsuite/tests/rts/InternalCounters.run InternalCounters [bad stdout] (normal) testsuite/tests/rts/T14702.run T14702 [exit code non-0] (threaded1) testsuite/tests/rts/nursery-chunks1.run nursery-chunks1 [exit code non-0] (threaded1) testsuite/tests/simplCore/should_compile/T3717.run T3717 [stderr mismatch] (optasm) testsuite/tests/simplCore/should_compile/T4930.run T4930 [stderr mismatch] (optasm) testsuite/tests/simplCore/should_compile/spec-inline.run spec-inline [stderr mismatch] (optasm) testsuite/tests/simplCore/should_compile/T4908.run T4908 [stderr mismatch] (optasm) testsuite/tests/simplCore/should_compile/T3055.run T3055 [bad stdout] (normal) testsuite/tests/simplCore/should_compile/T3772.run T3772 [bad stdout] (normal) testsuite/tests/simplCore/should_compile/T5327.run T5327 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T7360.run T7360 [stderr mismatch] (optasm) testsuite/tests/simplCore/should_compile/T7865.run T7865 [bad stdout] (normal) testsuite/tests/simplCore/should_compile/T8832.run T8832 [bad stdout] (normal) testsuite/tests/simplCore/should_compile/T12603.run T12603 [bad stdout] (normal) testsuite/tests/simplCore/should_compile/T13143.run T13143 [stderr mismatch] (optasm) testsuite/tests/simplCore/should_compile/T13340.run T13340 [bad exit code] (normal) testsuite/tests/simplCore/should_compile/T13543.run T13543 [stderr mismatch] (optasm) testsuite/tests/simplCore/should_compile/T15056.run T15056 [stderr mismatch] (normal) testsuite/tests/stranal/should_compile/T10694.run T10694 [stderr mismatch] (optasm) testsuite/tests/stranal/sigs/HyperStrUse.run HyperStrUse [stderr mismatch] (optasm) testsuite/tests/stranal/sigs/T12370.run T12370 [stderr mismatch] (optasm) testsuite/tests/stranal/sigs/CaseBinderCPR.run CaseBinderCPR [stderr mismatch] (optasm) testsuite/tests/th/TH_spliceE5_prof_ext.run TH_spliceE5_prof_ext [bad exit code] (normal) testsuite/tests/th/TH_spliceE5_prof.run TH_spliceE5_prof [bad exit code] (normal) testsuite/tests/typecheck/T13168/T13168.run T13168 [bad stderr] (normal) libraries/base/tests/memo001.run memo001 [bad exit code] (normal) libraries/base/tests/IO/T12010/T12010.run T12010 [bad exit code] (threaded1) libraries/ghc-compact/tests/compact_simple.run compact_simple [exit code non-0] (normal) libraries/ghc-compact/tests/compact_simple.run compact_simple [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_loop.run compact_loop [exit code non-0] (normal) libraries/ghc-compact/tests/compact_append.run compact_append [exit code non-0] (normal) libraries/ghc-compact/tests/compact_loop.run compact_loop [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_append.run compact_append [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_autoexpand.run compact_autoexpand [exit code non-0] (normal) libraries/ghc-compact/tests/compact_simple_array.run compact_simple_array [exit code non-0] (normal) libraries/ghc-compact/tests/compact_autoexpand.run compact_autoexpand [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_simple_array.run compact_simple_array [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_huge_array.run compact_huge_array [exit code non-0] (normal) libraries/ghc-compact/tests/compact_serialize.run compact_serialize [exit code non-0] (normal) libraries/ghc-compact/tests/compact_largemap.run compact_largemap [exit code non-0] (normal) libraries/ghc-compact/tests/compact_huge_array.run compact_huge_array [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_largemap.run compact_largemap [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_serialize.run compact_serialize [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_threads.run compact_threads [exit code non-0] (normal) libraries/ghc-compact/tests/compact_function.run compact_function [exit code non-0] (normal) libraries/ghc-compact/tests/compact_cycle.run compact_cycle [exit code non-0] (normal) libraries/ghc-compact/tests/compact_threads.run compact_threads [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_function.run compact_function [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_cycle.run compact_cycle [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_mutable.run compact_mutable [exit code non-0] (normal) libraries/ghc-compact/tests/compact_pinned.run compact_pinned [exit code non-0] (normal) libraries/ghc-compact/tests/compact_gc.run compact_gc [exit code non-0] (normal) libraries/ghc-compact/tests/compact_mutable.run compact_mutable [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_pinned.run compact_pinned [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_gc.run compact_gc [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_share.run compact_share [exit code non-0] (normal) libraries/ghc-compact/tests/compact_bench.run compact_bench [exit code non-0] (normal) libraries/ghc-compact/tests/compact_share.run compact_share [exit code non-0] (sanity) libraries/ghc-compact/tests/compact_bench.run compact_bench [exit code non-0] (sanity) libraries/process/tests/T3231.run T3231 [exit code non-0] (threaded1) libraries/process/tests/T3994.run T3994 [exit code non-0] (threaded1) libraries/stm/tests/stm056.run stm056 [exit code non-0] (threaded1) libraries/unix/tests/forkprocess01.run forkprocess01 [exit code non-0] (threaded1_ls) libraries/unix/tests/fdReadBuf001.run fdReadBuf001 [bad exit code] (ghci) testsuite/tests/rts/T7815.run T7815 [exit code non-0] (threaded1) Unexpected stat failures: testsuite/tests/callarity/perf/T3924.run T3924 [stat not good enough] (normal) testsuite/tests/perf/join_points/join004.run join004 [stat not good enough] (optasm) testsuite/tests/perf/should_run/T10359.run T10359 [stat not good enough] (normal) testsuite/tests/perf/join_points/join007.run join007 [stat not good enough] (optasm) testsuite/tests/perf/should_run/T14955.run T14955 [stat not good enough] (normal) testsuite/tests/perf/should_run/T4830.run T4830 [stat not good enough] (normal) testsuite/tests/perf/should_run/T3586.run T3586 [stat not good enough] (normal) testsuite/tests/perf/should_run/lazy-bs-alloc.run lazy-bs-alloc [stat not good enough] (normal) testsuite/tests/perf/should_run/T876.run T876 [stat not good enough] (normal) testsuite/tests/perf/should_run/T3738.run T3738 [stat not good enough] (normal) testsuite/tests/perf/should_run/MethSharing.run MethSharing [stat not good enough] (normal) testsuite/tests/perf/should_run/T5113.run T5113 [stat not good enough] (normal) testsuite/tests/perf/should_run/T4978.run T4978 [stat not good enough] (normal) testsuite/tests/perf/should_run/T5205.run T5205 [stat not good enough] (normal) testsuite/tests/perf/should_run/T4474a.run T4474a [stat not good enough] (normal) testsuite/tests/perf/should_run/T4474b.run T4474b [stat not good enough] (normal) testsuite/tests/perf/should_run/T5237.run T5237 [stat not good enough] (normal) testsuite/tests/perf/should_run/T4474c.run T4474c [stat not good enough] (normal) testsuite/tests/perf/should_run/T7257.run T7257 [stat not good enough] (normal) testsuite/tests/perf/should_run/Conversions.run Conversions [stat not good enough] (normal) testsuite/tests/perf/should_run/T5536.run T5536 [stat not good enough] (normal) testsuite/tests/perf/should_run/T7797.run T7797 [stat not good enough] (normal) testsuite/tests/perf/should_run/T5949.run T5949 [stat not good enough] (normal) testsuite/tests/perf/should_run/T4267.run T4267 [stat not good enough] (normal) testsuite/tests/perf/should_run/T7619.run T7619 [stat not good enough] (normal) testsuite/tests/perf/should_run/InlineByteArrayAlloc.run InlineByteArrayAlloc [stat not good enough] (normal) testsuite/tests/perf/should_run/InlineArrayAlloc.run InlineArrayAlloc [stat not good enough] (normal) testsuite/tests/perf/should_run/InlineCloneArrayAlloc.run InlineCloneArrayAlloc [stat not good enough] (normal) testsuite/tests/perf/should_run/T9203.run T9203 [stat not good enough] (normal) testsuite/tests/perf/should_run/T8472.run T8472 [stat not good enough] (normal) testsuite/tests/perf/should_run/T12996.run T12996 [stat not good enough] (normal) testsuite/tests/perf/should_run/T13001.run T13001 [stat not good enough] (normal) testsuite/tests/perf/should_run/T12990.run T12990 [stat not good enough] (normal) testsuite/tests/perf/should_run/DeriveNull.run DeriveNull [stat not good enough] (normal) testsuite/tests/perf/should_run/T14936.run T14936 [stat not good enough] (normal) testsuite/tests/perf/should_run/T13623.run T13623 [stat not good enough] (normal) testsuite/tests/perf/should_run/T15226.run T15226 [stat not good enough] (normal) testsuite/tests/perf/should_run/T15226a.run T15226a [stat not good enough] (normal) testsuite/tests/perf/space_leaks/space_leak_001.run space_leak_001 [stat not good enough] (normal) testsuite/tests/perf/space_leaks/T4334.run T4334 [stat not good enough] (normal) testsuite/tests/perf/should_run/T5549.run T5549 [stat not good enough] (normal) libraries/base/tests/T8766.run T8766 [stat not good enough] (normal) libraries/base/tests/T13191.run T13191 [stat not good enough] (normal) Framework failures: testsuite/tests/perf/haddock/haddock.base.run haddock.base [normal] (extra_file does not exist: ../../../../libraries/base/dist-install/haddock.t) testsuite/tests/perf/haddock/haddock.Cabal.run haddock.Cabal [normal] (extra_file does not exist: ../../../../libraries/Cabal/Cabal/dist-install/haddock.t) testsuite/tests/perf/haddock/haddock.compiler.run haddock.compiler [normal] (extra_file does not exist: ../../../../compiler/stage2/haddock.t) ```

Note: We're still skipping many many tests for some reason. For reference, a ./validate run from today, on GHC's master branch gives the following numbers:

SUMMARY for test run started at Tue Sep  4 15:34:20 2018 UTC
 0:17:11 spent to go through
    6537 total tests, which gave rise to
   25537 test cases, of which
   18644 were skipped

      40 had missing libraries
    6765 expected passes
      89 expected failures

       0 caused framework failures
       0 caused framework warnings
       0 unexpected passes
       0 unexpected failures
       0 unexpected stat failures
alpmestan commented 6 years ago

My differential has been merged: https://github.com/ghc/ghc/commit/2254912036597cee3f1c2544320a35d1115c1e6e

Meanwhile, I looked at #15570 and #15286 and it turns out removing those -O0 apparently makes more tests pass! In particular, all the stats failures but one are gone (makes sense), along with a little over 10 unexpected failures. I've got a PR up at #674. An entire test rule run also takes a lot less time now :-)

New test results ``` SUMMARY for test run started at Tue Sep 4 22:07:26 2018 CEST 0:18:56 spent to go through 6536 total tests, which gave rise to 24042 test cases, of which 17523 were skipped 27 had missing libraries 6225 expected passes 96 expected failures 3 caused framework failures 0 caused framework warnings 1 unexpected passes 169 unexpected failures 1 unexpected stat failures ```

Many, many of the remaining failures are due to ghci printing things differently (e.g 0 vs integer-...:Some.Module.S# 0) and therefore giving semantically equivalent but syntactically different outputs. I have seen #13243 which is about the same problem when running tests directly with make instead of through validate, but there isn't any conclusion about why this happens, even less about how to fix it. Somehow validate doesn't have this problem though. I guess I'll be the one figuring this out. But enough for today (and I will likely have to work on a few other things too during the rest of the week).

snowleopard commented 6 years ago

Great work, @alpmestan!

I'll try to figure out why ghci prints different results and report back if I find anything.

alpmestan commented 6 years ago

OK; make sure you use --flavour=perf with the test target.

alpmestan commented 6 years ago

https://github.com/snowleopard/hadrian/pull/677 makes all the ghc-compact tests pass.

alpmestan commented 6 years ago

With https://github.com/snowleopard/hadrian/pull/684, I get:

SUMMARY for test run started at Tue Sep 11 17:29:41 2018 CEST
 0:16:40 spent to go through
    6538 total tests, which gave rise to
   24053 test cases, of which
   17532 were skipped

      27 had missing libraries
    6315 expected passes
      94 expected failures

       3 caused framework failures
       0 caused framework warnings
       1 unexpected passes
      83 unexpected failures
       1 unexpected stat failures

This is looking better and better.

snowleopard commented 6 years ago

@alpmestan Awesome!

What about 17532 were skipped -- is this normal?

alpmestan commented 6 years ago

See my comment above:

Note: We're still skipping many many tests for some reason. For reference, a ./validate run from today, on GHC's master branch gives the following numbers:

SUMMARY for test run started at Tue Sep  4 15:34:20 2018 UTC
 0:17:11 spent to go through
    6537 total tests, which gave rise to
   25537 test cases, of which
   18644 were skipped

      40 had missing libraries
   6765 expected passes
     89 expected failures

      0 caused framework failures
      0 caused framework warnings
      0 unexpected passes
      0 unexpected failures
      0 unexpected stat failures

You can see that the normal validate script skips many tests too. Many tests are restricted to be executed only on some arch, some os, some way, some [something] that'll cause it to be skipped in many situations. We're still missing some tests and probably skipping too many of them, but we're somewhat close to what the make build system does, aren't we?

simonmar commented 6 years ago

Hi @alpmestan, this might not be the right place but I wanted to ask about workflows for the testsuite. Very often I do something like

cd testsuite/tests/codeGen/should_run
make TEST=cgrun001 WAY=optasm CLEANUP=0 EXTRA_HC_OPTS=-debug

to run a single test with some custom options. This is really convenient (and fast). Will we have something equally convenient with Hadrian, and will it be just as fast?

alpmestan commented 6 years ago

@simonmar We don't do anything $PWD-dependent for now. We do have --only=... to specify a specific test, --test-way to specify a particular test way, but I don't think we have anything to specify extra options right now, even though I suspect you can achieve this using --config.

(for reference:

Usage: hadrian [options] [target] ...
Options:
  -a FULL=SHORT, --abbrev=FULL=SHORT
                              Use abbreviation in status messages.
  --no-build                  Don't build anything.
  -C DIRECTORY, --directory=DIRECTORY
                              Change to DIRECTORY before doing anything.
  --color, --colour           Colorize the output.
  --no-color, --no-colour     Don't colorize the output.
  -d[FILE], --debug[=FILE]    Print lots of debugging information.
  --demo                      Run in demo mode.
  --digest                    Files change when digest changes.
  --digest-and                Files change when modtime and digest change.
  --digest-and-input          Files change on modtime (and digest for inputs).
  --digest-or                 Files change when modtime or digest change.
  --digest-not                Files change when modtime changes.
  --exception                 Throw exceptions directly.
  --flush=N                   Flush metadata every N seconds.
  --never-flush               Never explicitly flush metadata.
  -h, --help                  Print this message and exit.
  -j[N], --jobs[=N]           Allow N jobs/threads at once [default CPUs].
  -k, --keep-going            Keep going when some targets can't be made.
  -l, --lint                  Perform limited validation after the run.
  --lint-fsatrace             Use fsatrace to do validation.
  --no-lint                   Turn off --lint.
  --live[=FILE]               List the files that are live [to live.txt].
  -m PREFIX, --metadata=PREFIX
                              Prefix for storing metadata files.
  --numeric-version           Print just the version number and exit.
  --skip-commands             Try and avoid running external programs.
  --rebuild[=PATTERN]         Rebuild matching files.
  --no-rebuild[=PATTERN]      Rebuild matching files if necessary.
  --skip[=PATTERN]            Don't rebuild matching files this run.
  -r[FILE], --report[=FILE], --profile[=FILE]
                              Write out profiling information [to report.html].
  --no-reports                Turn off --report.
  --rule-version=VERSION      Version of the build rules.
  --no-rule-version           Ignore the build rules version.
  -s, --silent                Don't print anything.
  --sleep                     Sleep for a second before building.
  -S, --no-keep-going, --stop
                              Turns off -k.
  --storage                   Write a storage log.
  -p[N], --progress[=N]       Show progress messages [every N secs, default 5].
  --no-progress               Don't show progress messages.
  -q, --quiet                 Print less (pass repeatedly for even less).
  --no-time                   Don't print build time.
  --timings                   Print phase timings.
  -V, --verbose, --trace      Print more (pass repeatedly for even more).
  -v, --version               Print the version number and exit.
  -w, --print-directory       Print the current directory.
  --no-print-directory        Turn off -w, even if it was turned on implicitly.
  -c, --configure             Run the boot and configure scripts (if you do not want to run them manually).
  -o[BUILD_ROOT], --build-root[=BUILD_ROOT]
                              Where to store build artifacts. (Default _build).
  --flavour[=FLAVOUR]         Build flavour (Default, Devel1, Devel2, Perf, Prof, Quick or Quickest).
  --freeze1                   Freeze Stage1 GHC.
  --integer-simple            Build GHC with integer-simple library.
  --progress-colour[=MODE]    Use colours in progress info (Never, Auto or Always).
  --progress-info[=STYLE]     Progress info style (None, Brief, Normal or Unicorn).
  --split-objects             Generate split objects (requires a full clean rebuild).
  --test-compiler[=TEST_COMPILER]
                              Use given compiler [Default=stage2].
  --test-config-file[=CONFIG_FILE]
                              congiguration file for testsuite. Default=testsuite/config/ghc
  --config[=EXTRA_TEST_CONFIG]
                              Configurations to run test, in key=value format.
  --summary-junit[=TEST_SUMMARY_JUNIT]
                              Output testsuite summary in JUnit format.
  --only[=TESTS]              Test cases to run.
  --only-perf                 Only run performance tests.
  --skip-perf                 Skip performance tests.
  --test-speed[=SPEED]        fast, slow or normal. Normal by default
  --summary[=TEST_SUMMARY]    Where to output the test summary file.
  --test-verbose[=TEST_VERBOSE]
                              A verbosity value between 0 and 5. 0 is silent, 4 and higher activates extra output.
  --test-way[=TEST_WAY]       only run these ways

)

simonmar commented 6 years ago

In that case can I request that support for this be added? It's really important when developing to be able to work on a single test, or a single directory of tests, and to be able to customize options, compilers and cleanup behaviour. I think we'll need this before we can drop the old build system.

alpmestan commented 6 years ago

@simonmar I opened #685 and #686 to remember about this (which involves two distinct improvements, AFAICT) and track the progress. Feel free to subscribe to those if you want to get a notification when this gets implemented. Or to even leave comments if you have anything to add to those issues.

simonmar commented 6 years ago

Thanks @alpmestan. To add a bit more to this:

alpmestan commented 6 years ago

@simonmar As I pointed out in earlier comments, we can already specify a particular WAY, through --test-way. And the test compiler with --test-compiler. We however don't have:

Thanks for the input! It's helpful to have very precise TODOs like this :-)

alpmestan commented 6 years ago

The PR at https://github.com/snowleopard/hadrian/pull/687 fixes another batch of failures, in addition to solving a rather important problem. With this patch I get:

SUMMARY for test run started at Wed Sep 12 16:58:01 2018 CEST
 0:27:00 spent to go through
    6538 total tests, which gave rise to
   24053 test cases, of which
   17532 were skipped

      27 had missing libraries
    6341 expected passes
      94 expected failures

       3 caused framework failures
       0 caused framework warnings
       1 unexpected passes
      57 unexpected failures
       1 unexpected stat failures

Remaining failures:

Unexpected results from:
TEST="InternalCounters T10294 T10294a T10420 T10989 T12567a T12962 T13168 T13414 T13702 T14931 T2552 T2991 T3103 T5363 T5559 T680 T8209 T8242 T8430 T9203 T9208 T9293 TH_spliceE5_prof TH_spliceE5_prof_ext cabal01 cabal03 cabal04 conc069 conc069a conc070 fdReadBuf001 ghci057 haddock.Cabal haddock.base haddock.compiler memo001 memo002 plugin-recomp-change plugin-recomp-change-prof plugin-recomp-flags plugin-recomp-impure plugin-recomp-pure plugins01 plugins07 plugins09 plugins11 plugins12 plugins13 plugins14 plugins15 prof-doc-fib prof-doc-last profinline001 read041 scc001 scc002 scc003 scc005"
...
Unexpected passes:
   testsuite/tests/stranal/should_compile/T9208.run  T9208 [unexpected] (optasm)

Unexpected failures:
   testsuite/tests/cabal/cabal01/cabal01.run               cabal01 [bad exit code] (normal)
   testsuite/tests/cabal/cabal03/cabal03.run               cabal03 [bad exit code] (normal)
   testsuite/tests/cabal/cabal04/cabal04.run               cabal04 [bad exit code] (normal)
   testsuite/tests/concurrent/should_run/conc069.run       conc069 [bad exit code] (ghci)
   testsuite/tests/concurrent/should_run/conc069a.run      conc069a [bad exit code] (ghci)
   testsuite/tests/concurrent/should_run/conc070.run       conc070 [bad stdout] (ghci)
   testsuite/tests/dynlibs/T13702.run                      T13702 [bad exit code] (normal)
   testsuite/tests/ghci/scripts/ghci057.run                ghci057 [bad stdout] (ghci)
   testsuite/tests/ghci/scripts/T9293.run                  T9293 [bad stdout] (ghci)
   testsuite/tests/ghci/scripts/T10989.run                 T10989 [bad stderr] (ghci)
   testsuite/tests/hpc/T2991.run                           T2991 [exit code non-0] (hpc)
   testsuite/tests/parser/should_compile/read041.run       read041 [exit code non-0] (normal)
   testsuite/tests/parser/should_fail/T8430.run            T8430 [stderr mismatch] (normal)
   testsuite/tests/parser/should_fail/T13414.run           T13414 [stderr mismatch] (normal)
   testsuite/tests/perf/haddock/haddock.base.run           haddock.base [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.base.run/haddock.t'] (normal)
   testsuite/tests/perf/haddock/haddock.Cabal.run          haddock.Cabal [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.Cabal.run/haddock.t'] (normal)
   testsuite/tests/perf/haddock/haddock.compiler.run       haddock.compiler [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.compiler.run/haddock.t'] (normal)
   testsuite/tests/plugins/plugins01.run                   plugins01 [bad exit code] (normal)
   testsuite/tests/plugins/plugins07.run                   plugins07 [bad exit code] (normal)
   testsuite/tests/plugins/plugins09.run                   plugins09 [bad exit code] (normal)
   testsuite/tests/plugins/plugins11.run                   plugins11 [bad exit code] (normal)
   testsuite/tests/plugins/plugins12.run                   plugins12 [bad exit code] (normal)
   testsuite/tests/plugins/plugins13.run                   plugins13 [bad exit code] (normal)
   testsuite/tests/plugins/plugins14.run                   plugins14 [bad exit code] (normal)
   testsuite/tests/plugins/plugins15.run                   plugins15 [bad exit code] (normal)
   testsuite/tests/plugins/T10420.run                      T10420 [bad exit code] (normal)
   testsuite/tests/plugins/T10294.run                      T10294 [bad exit code] (normal)
   testsuite/tests/plugins/T10294a.run                     T10294a [bad exit code] (normal)
   testsuite/tests/plugins/T12567a.run                     T12567a [bad exit code] (normal)
   testsuite/tests/plugins/plugin-recomp-pure.run          plugin-recomp-pure [bad exit code] (normal)
   testsuite/tests/plugins/plugin-recomp-impure.run        plugin-recomp-impure [bad exit code] (normal)
   testsuite/tests/plugins/plugin-recomp-change.run        plugin-recomp-change [bad exit code] (normal)
   testsuite/tests/plugins/plugin-recomp-flags.run         plugin-recomp-flags [bad exit code] (normal)
   testsuite/tests/plugins/plugin-recomp-change-prof.run   plugin-recomp-change-prof [bad exit code] (normal)
   testsuite/tests/profiling/should_compile/T14931.run     T14931 [bad exit code] (normal)
   testsuite/tests/profiling/should_run/scc001.run         scc001 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/scc002.run         scc002 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/scc003.run         scc003 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/scc005.run         scc005 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/T680.run           T680 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/T2552.run          T2552 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/prof-doc-fib.run   prof-doc-fib [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/prof-doc-last.run  prof-doc-last [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/T5559.run          T5559 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/profinline001.run  profinline001 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/T5363.run          T5363 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/T12962.run         T12962 [bad exit code] (ghci-ext-prof)
   testsuite/tests/rename/should_compile/T3103/T3103.run   T3103 [exit code non-0] (normal)
   testsuite/tests/rts/T8209.run                           T8209 [bad stderr] (ghci)
   testsuite/tests/rts/T8242.run                           T8242 [bad stderr] (ghci)
   testsuite/tests/rts/InternalCounters.run                InternalCounters [bad stdout] (normal)
   testsuite/tests/th/TH_spliceE5_prof_ext.run             TH_spliceE5_prof_ext [bad exit code] (normal)
   testsuite/tests/th/TH_spliceE5_prof.run                 TH_spliceE5_prof [bad exit code] (normal)
   testsuite/tests/typecheck/T13168/T13168.run             T13168 [bad stderr] (normal)
   libraries/base/tests/memo001.run                        memo001 [exit code non-0] (normal)
   libraries/base/tests/memo002.run                        memo002 [exit code non-0] (normal)
   libraries/unix/tests/fdReadBuf001.run                   fdReadBuf001 [bad exit code] (ghci)

Unexpected stat failures:
   testsuite/tests/perf/should_run/T9203.run  T9203 [stat too good] (normal)

Framework failures:
   testsuite/tests/perf/haddock/haddock.base.run      haddock.base [normal] (extra_file does not exist: ../../../../libraries/base/dist-install/haddock.t)
   testsuite/tests/perf/haddock/haddock.Cabal.run     haddock.Cabal [normal] (extra_file does not exist: ../../../../libraries/Cabal/Cabal/dist-install/haddock.t)
   testsuite/tests/perf/haddock/haddock.compiler.run  haddock.compiler [normal] (extra_file does not exist: ../../../../compiler/stage2/haddock.t)
alpmestan commented 6 years ago

Aaaaand we're down to 46 unexpected failures with https://github.com/snowleopard/hadrian/pull/688:

Unexpected results from:
TEST="InternalCounters T10294 T10294a T10420 T12567a T12962 T13168 T13702 T14931 T2552 T5363 T5559 T680 T8209 T8242 T9203 T9208 T9293 TH_spliceE5_prof TH_spliceE5_prof_ext conc069 conc069a conc070 fdReadBuf001 ghci057 haddock.Cabal haddock.base haddock.compiler plugin-recomp-change plugin-recomp-change-prof plugin-recomp-flags plugin-recomp-impure plugin-recomp-pure plugins01 plugins07 plugins09 plugins11 plugins12 plugins13 plugins14 plugins15 prof-doc-fib prof-doc-last profinline001 scc001 scc002 scc003 scc005"

SUMMARY for test run started at Wed Sep 12 21:50:48 2018 CEST
 0:20:42 spent to go through
    6538 total tests, which gave rise to
   24053 test cases, of which
   17532 were skipped

      27 had missing libraries
    6352 expected passes
      94 expected failures

       3 caused framework failures
       0 caused framework warnings
       1 unexpected passes
      46 unexpected failures
       1 unexpected stat failures

Unexpected passes:
   testsuite/tests/stranal/should_compile/T9208.run  T9208 [unexpected] (optasm)

Unexpected failures:
   testsuite/tests/concurrent/should_run/conc069.run       conc069 [bad exit code] (ghci)
   testsuite/tests/concurrent/should_run/conc069a.run      conc069a [bad exit code] (ghci)
   testsuite/tests/concurrent/should_run/conc070.run       conc070 [bad stdout] (ghci)
   testsuite/tests/dynlibs/T13702.run                      T13702 [bad exit code] (normal)
   testsuite/tests/ghci/scripts/ghci057.run                ghci057 [bad stdout] (ghci)
   testsuite/tests/ghci/scripts/T9293.run                  T9293 [bad stdout] (ghci)
   testsuite/tests/perf/haddock/haddock.base.run           haddock.base [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.base.run/haddock.t'] (normal)
   testsuite/tests/perf/haddock/haddock.Cabal.run          haddock.Cabal [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.Cabal.run/haddock.t'] (normal)
   testsuite/tests/perf/haddock/haddock.compiler.run       haddock.compiler [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.compiler.run/haddock.t'] (normal)
   testsuite/tests/plugins/plugins01.run                   plugins01 [bad exit code] (normal)
   testsuite/tests/plugins/plugins07.run                   plugins07 [bad exit code] (normal)
   testsuite/tests/plugins/plugins09.run                   plugins09 [bad exit code] (normal)
   testsuite/tests/plugins/plugins11.run                   plugins11 [bad exit code] (normal)
   testsuite/tests/plugins/plugins12.run                   plugins12 [bad exit code] (normal)
   testsuite/tests/plugins/plugins13.run                   plugins13 [bad exit code] (normal)
   testsuite/tests/plugins/plugins14.run                   plugins14 [bad exit code] (normal)
   testsuite/tests/plugins/plugins15.run                   plugins15 [bad exit code] (normal)
   testsuite/tests/plugins/T10420.run                      T10420 [bad exit code] (normal)
   testsuite/tests/plugins/T10294.run                      T10294 [bad exit code] (normal)
   testsuite/tests/plugins/T10294a.run                     T10294a [bad exit code] (normal)
   testsuite/tests/plugins/T12567a.run                     T12567a [bad exit code] (normal)
   testsuite/tests/plugins/plugin-recomp-pure.run          plugin-recomp-pure [bad exit code] (normal)
   testsuite/tests/plugins/plugin-recomp-impure.run        plugin-recomp-impure [bad exit code] (normal)
   testsuite/tests/plugins/plugin-recomp-flags.run         plugin-recomp-flags [bad exit code] (normal)
   testsuite/tests/plugins/plugin-recomp-change.run        plugin-recomp-change [bad exit code] (normal)
   testsuite/tests/plugins/plugin-recomp-change-prof.run   plugin-recomp-change-prof [bad exit code] (normal)
   testsuite/tests/profiling/should_compile/T14931.run     T14931 [bad exit code] (normal)
   testsuite/tests/profiling/should_run/scc001.run         scc001 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/scc002.run         scc002 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/scc003.run         scc003 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/scc005.run         scc005 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/T680.run           T680 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/T2552.run          T2552 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/prof-doc-fib.run   prof-doc-fib [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/prof-doc-last.run  prof-doc-last [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/T5559.run          T5559 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/profinline001.run  profinline001 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/T5363.run          T5363 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/T12962.run         T12962 [bad exit code] (ghci-ext-prof)
   testsuite/tests/rts/T8209.run                           T8209 [bad stderr] (ghci)
   testsuite/tests/rts/T8242.run                           T8242 [bad stderr] (ghci)
   testsuite/tests/rts/InternalCounters.run                InternalCounters [bad stdout] (normal)
   testsuite/tests/th/TH_spliceE5_prof_ext.run             TH_spliceE5_prof_ext [bad exit code] (normal)
   testsuite/tests/th/TH_spliceE5_prof.run                 TH_spliceE5_prof [bad exit code] (normal)
   testsuite/tests/typecheck/T13168/T13168.run             T13168 [bad stderr] (normal)
   libraries/unix/tests/fdReadBuf001.run                   fdReadBuf001 [bad exit code] (ghci)

Unexpected stat failures:
   testsuite/tests/perf/should_run/T9203.run  T9203 [stat too good] (normal)

Framework failures:
   testsuite/tests/perf/haddock/haddock.base.run      haddock.base [normal] (extra_file does not exist: ../../../../libraries/base/dist-install/haddock.t)
   testsuite/tests/perf/haddock/haddock.Cabal.run     haddock.Cabal [normal] (extra_file does not exist: ../../../../libraries/Cabal/Cabal/dist-install/haddock.t)
   testsuite/tests/perf/haddock/haddock.compiler.run  haddock.compiler [normal] (extra_file does not exist: ../../../../compiler/stage2/haddock.t)
alpmestan commented 6 years ago

https://github.com/snowleopard/hadrian/pull/689 and https://phabricator.haskell.org/D5146 solve some more. Down to 42 unexpected failures.

Unexpected results from:
TEST="T10294 T10294a T10420 T12567a T12962 T13168 T13702 T14931 T2552 T5363 T5559 T680 T9203 T9208 T9293 TH_spliceE5_prof TH_spliceE5_prof_ext bkpcabal01 bkpcabal02 bkpcabal03 ghci057 haddock.Cabal haddock.base haddock.compiler plugin-recomp-change plugin-recomp-change-prof plugin-recomp-flags plugin-recomp-impure plugin-recomp-pure plugins01 plugins07 plugins09 plugins11 plugins12 plugins13 plugins14 plugins15 prof-doc-fib prof-doc-last profinline001 scc001 scc002 scc003 scc005"

SUMMARY for test run started at Thu Sep 13 09:04:24 2018 CEST
 0:15:55 spent to go through
    6539 total tests, which gave rise to
   24054 test cases, of which
   17532 were skipped

      27 had missing libraries
    6357 expected passes
      94 expected failures

       3 caused framework failures
       0 caused framework warnings
       1 unexpected passes
      42 unexpected failures
       1 unexpected stat failures

Unexpected passes:
   testsuite/tests/stranal/should_compile/T9208.run  T9208 [unexpected] (optasm)

Unexpected failures:
   testsuite/tests/backpack/cabal/bkpcabal02/bkpcabal02.run  bkpcabal02 [bad exit code] (normal)
   testsuite/tests/backpack/cabal/bkpcabal03/bkpcabal03.run  bkpcabal03 [bad exit code] (normal)
   testsuite/tests/backpack/cabal/bkpcabal01/bkpcabal01.run  bkpcabal01 [bad stdout] (normal)
   testsuite/tests/dynlibs/T13702.run                        T13702 [bad exit code] (normal)
   testsuite/tests/ghci/scripts/ghci057.run                  ghci057 [bad stdout] (ghci)
   testsuite/tests/ghci/scripts/T9293.run                    T9293 [bad stdout] (ghci)
   testsuite/tests/perf/haddock/haddock.base.run             haddock.base [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.base.run/haddock.t'] (normal)
   testsuite/tests/perf/haddock/haddock.Cabal.run            haddock.Cabal [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.Cabal.run/haddock.t'] (normal)
   testsuite/tests/perf/haddock/haddock.compiler.run         haddock.compiler [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.compiler.run/haddock.t'] (normal)
   testsuite/tests/plugins/plugins01.run                     plugins01 [bad exit code] (normal)
   testsuite/tests/plugins/plugins07.run                     plugins07 [bad exit code] (normal)
   testsuite/tests/plugins/plugins09.run                     plugins09 [bad exit code] (normal)
   testsuite/tests/plugins/plugins11.run                     plugins11 [bad exit code] (normal)
   testsuite/tests/plugins/plugins12.run                     plugins12 [bad exit code] (normal)
   testsuite/tests/plugins/plugins13.run                     plugins13 [bad exit code] (normal)
   testsuite/tests/plugins/plugins14.run                     plugins14 [bad exit code] (normal)
   testsuite/tests/plugins/plugins15.run                     plugins15 [bad exit code] (normal)
   testsuite/tests/plugins/T10420.run                        T10420 [bad exit code] (normal)
   testsuite/tests/plugins/T10294.run                        T10294 [bad exit code] (normal)
   testsuite/tests/plugins/T10294a.run                       T10294a [bad exit code] (normal)
   testsuite/tests/plugins/T12567a.run                       T12567a [bad exit code] (normal)
   testsuite/tests/plugins/plugin-recomp-pure.run            plugin-recomp-pure [bad exit code] (normal)
   testsuite/tests/plugins/plugin-recomp-impure.run          plugin-recomp-impure [bad exit code] (normal)
   testsuite/tests/plugins/plugin-recomp-flags.run           plugin-recomp-flags [bad exit code] (normal)
   testsuite/tests/plugins/plugin-recomp-change.run          plugin-recomp-change [bad exit code] (normal)
   testsuite/tests/plugins/plugin-recomp-change-prof.run     plugin-recomp-change-prof [bad exit code] (normal)
   testsuite/tests/profiling/should_compile/T14931.run       T14931 [bad exit code] (normal)
   testsuite/tests/profiling/should_run/scc001.run           scc001 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/scc002.run           scc002 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/scc003.run           scc003 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/scc005.run           scc005 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/T680.run             T680 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/T2552.run            T2552 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/prof-doc-fib.run     prof-doc-fib [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/prof-doc-last.run    prof-doc-last [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/T5559.run            T5559 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/profinline001.run    profinline001 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/T5363.run            T5363 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/T12962.run           T12962 [bad exit code] (ghci-ext-prof)
   testsuite/tests/th/TH_spliceE5_prof_ext.run               TH_spliceE5_prof_ext [bad exit code] (normal)
   testsuite/tests/th/TH_spliceE5_prof.run                   TH_spliceE5_prof [bad exit code] (normal)
   testsuite/tests/typecheck/T13168/T13168.run               T13168 [bad stderr] (normal)

Unexpected stat failures:
   testsuite/tests/perf/should_run/T9203.run  T9203 [stat too good] (normal)

Framework failures:
   testsuite/tests/perf/haddock/haddock.base.run      haddock.base [normal] (extra_file does not exist: ../../../../libraries/base/dist-install/haddock.t)
   testsuite/tests/perf/haddock/haddock.Cabal.run     haddock.Cabal [normal] (extra_file does not exist: ../../../../libraries/Cabal/Cabal/dist-install/haddock.t)
   testsuite/tests/perf/haddock/haddock.compiler.run  haddock.compiler [normal] (extra_file does not exist: ../../../../compiler/stage2/haddock.t)

I can already smell the wonderful perfume of a completely green hadrian tesŧ running in our future CI jobs in the main GHC repo. :nose: :+1:

alpmestan commented 6 years ago

Earlier today, I ran the test rule (with the perf flavour as usual) and... things went pretty well.

SUMMARY for test run started at Tue Oct 23 11:07:46 2018 CEST
 0:15:00 spent to go through
    6601 total tests, which gave rise to
   25634 test cases, of which
   19054 were skipped

      27 had missing libraries
    6431 expected passes
      89 expected failures

       3 caused framework failures
       0 caused framework warnings
       1 unexpected passes
      31 unexpected failures
       1 unexpected stat failures

I looked at all the failures and managed to classify them in a few categories.

Require a GHC built with -dynamic

   testsuite/tests/typecheck/should_compile/T11462.run                T11462 [exit code non-0] (normal)
   testsuite/tests/typecheck/should_compile/T11525.run                T11525 [exit code non-0] (normal)
   testsuite/tests/th/TH_linker/path_with_commas.run                  path_with_commas [bad exit code] (normal) -- not entirely sure about this one
   testsuite/tests/plugins/plugins05.run                              plugins05 [exit code non-0] (dyn)
   testsuite/tests/plugins/plugins06.run                              plugins06 [exit code non-0] (dyn)

This will be fixed once the joint work with David lands (Cabal + ghc + hadrian patch). See #691.

Cannot locate libffi or libHSrts

   testsuite/tests/driver/dynHelloWorld.run                           dynHelloWorld [exit code non-0] (dyn)
   testsuite/tests/dynlibs/T5373.run                                  T5373 [bad exit code] (normal)
   testsuite/tests/dynlibs/T3807.run                                  T3807 [bad exit code] (normal)
   testsuite/tests/dynlibs/T13702.run                                 T13702 [bad exit code] (normal)

We don't build ghc-iserv-prof

The make build system builds a -prof version of ghc-iserv which it calls ghc-iserv-prof, and this is required by a few tests. I'm going to look into how we can produce two executables from just one package...

   testsuite/tests/profiling/should_run/scc001.run                    scc001 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/scc002.run                    scc002 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/scc003.run                    scc003 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/scc005.run                    scc005 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/T680.run                      T680 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/T2552.run                     T2552 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/prof-doc-fib.run              prof-doc-fib [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/prof-doc-last.run             prof-doc-last [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/T5559.run                     T5559 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/profinline001.run             profinline001 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/T5363.run                     T5363 [bad exit code] (ghci-ext-prof)
   testsuite/tests/profiling/should_run/T12962.run                    T12962 [bad exit code] (ghci-ext-prof)
   testsuite/tests/th/TH_spliceE5_prof_ext.run                        TH_spliceE5_prof_ext [bad exit code] (normal)

Less output than expected

Those in generally look benign, but I have no idea why the output differs.

   testsuite/tests/annotations/should_compile/th/annth_compunits.run  annth_compunits [bad exit code] (normal)
   testsuite/tests/annotations/should_compile/th/annth_make.run       annth_make [bad stdout] (normal)
   testsuite/tests/ghci/scripts/ghci057.run                           ghci057 [bad stdout] (ghci)
   testsuite/tests/ghci/scripts/T9293.run                             T9293 [bad stdout] (ghci)
   testsuite/tests/typecheck/T13168/T13168.run                        T13168 [bad stderr] (normal)

Bad handling of -osuf + -prof options ?

   testsuite/tests/th/TH_spliceE5_prof.run                            TH_spliceE5_prof [bad exit code] (normal)

haddock perf failures

We quite likely don't fit with the expected inplace layout, something like that?

   testsuite/tests/perf/haddock/haddock.base.run                      haddock.base [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.base.run/haddock.t'] (normal)
   testsuite/tests/perf/haddock/haddock.Cabal.run                     haddock.Cabal [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.Cabal.run/haddock.t'] (normal)
   testsuite/tests/perf/haddock/haddock.compiler.run                  haddock.compiler [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.compiler.run/haddock.t'] (normal)
snowleopard commented 6 years ago

Thanks @alpmestan -- looks like we're almost there!

alpmestan commented 6 years ago

With https://phabricator.haskell.org/D5255 I get:

Unexpected results from:
TEST="T11462 T11525 T13168 T13702 T3807 T5373 T9203 T9208 T9293 TH_spliceE5_prof annth_compunits annth_make dynHelloWorld ghci057 haddock.Cabal haddock.base haddock.compiler path_with_commas plugins05 plugins06"

SUMMARY for test run started at Wed Oct 24 13:53:33 2018 CEST
 0:14:42 spent to go through
    6601 total tests, which gave rise to
   25634 test cases, of which
   19054 were skipped

      27 had missing libraries
    6444 expected passes
      89 expected failures

       3 caused framework failures
       0 caused framework warnings
       1 unexpected passes
      18 unexpected failures
       1 unexpected stat failures
alpmestan commented 6 years ago

Test run against ghc/ghc@1a3b9bd0b674ad16a41b942c738b8f34564bcd8d:

Unexpected results from:
TEST="T11462 T11525 T13168 T13702 T3807 T5373 T9203 T9208 T9293 TH_spliceE5_prof annth_compunits annth_make dynHelloWorld frontend01 ghci057 haddock.Cabal haddock.base haddock.compiler path_with_commas plugins09 plugins11"

SUMMARY for test run started at Mon Nov  5 11:32:48 2018 CET
 0:13:08 spent to go through
    6656 total tests, which gave rise to
   25841 test cases, of which
   19176 were skipped

      28 had missing libraries
    6526 expected passes
      90 expected failures

       3 caused framework failures
       0 caused framework warnings
       1 unexpected passes
      19 unexpected failures
       1 unexpected stat failures

Unexpected passes:
   testsuite/tests/stranal/should_compile/T9208.run  T9208 [unexpected] (optasm)

Unexpected failures:
   testsuite/tests/annotations/should_compile/th/annth_make.run       annth_make [bad exit code] (normal)
   testsuite/tests/annotations/should_compile/th/annth_compunits.run  annth_compunits [bad stdout] (normal)
   testsuite/tests/driver/dynHelloWorld.run                           dynHelloWorld [exit code non-0] (dyn)
   testsuite/tests/dynlibs/T5373.run                                  T5373 [bad exit code] (normal)
   testsuite/tests/dynlibs/T3807.run                                  T3807 [bad exit code] (normal)
   testsuite/tests/dynlibs/T13702.run                                 T13702 [bad exit code] (normal)
   testsuite/tests/ghci/scripts/ghci057.run                           ghci057 [bad stdout] (ghci)
   testsuite/tests/ghci/scripts/T9293.run                             T9293 [bad stdout] (ghci)
   testsuite/tests/perf/haddock/haddock.base.run                      haddock.base [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.base.run/haddock.t'] (normal)
   testsuite/tests/perf/haddock/haddock.Cabal.run                     haddock.Cabal [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.Cabal.run/haddock.t'] (normal)
   testsuite/tests/perf/haddock/haddock.compiler.run                  haddock.compiler [[Errno 2] No such file or directory: 'testsuite/tests/perf/haddock/haddock.compiler.run/haddock.t'] (normal)
   testsuite/tests/plugins/plugins09.run                              plugins09 [bad stdout] (normal)
   testsuite/tests/plugins/plugins11.run                              plugins11 [bad stdout] (normal)
   testsuite/tests/plugins/frontend01.run                             frontend01 [bad stdout] (normal)
   testsuite/tests/th/TH_spliceE5_prof.run                            TH_spliceE5_prof [bad exit code] (normal)
   testsuite/tests/th/TH_linker/path_with_commas.run                  path_with_commas [bad exit code] (normal)
   testsuite/tests/typecheck/T13168/T13168.run                        T13168 [bad stderr] (normal)
   testsuite/tests/typecheck/should_compile/T11462.run                T11462 [exit code non-0] (normal)
   testsuite/tests/typecheck/should_compile/T11525.run                T11525 [exit code non-0] (normal)

Unexpected stat failures:
   testsuite/tests/perf/should_run/T9203.run  T9203 [stat too good] (normal)

Framework failures:
   testsuite/tests/perf/haddock/haddock.base.run      haddock.base [normal] (extra_file does not exist: ../../../../libraries/base/dist-install/haddock.t)
   testsuite/tests/perf/haddock/haddock.Cabal.run     haddock.Cabal [normal] (extra_file does not exist: ../../../../libraries/Cabal/Cabal/dist-install/haddock.t)
   testsuite/tests/perf/haddock/haddock.compiler.run  haddock.compiler [normal] (extra_file does not exist: ../../../../compiler/stage2/haddock.t)
snowleopard commented 5 years ago

I believe this issue is out of date, so closing.