search

LuaJIT / LuaJIT-test-cleanup

Cleanup Workspace for LuaJIT tests
46 stars 19 forks source link

Shake things up a bit around here #7

Closed corsix closed 8 years ago

corsix commented 8 years ago

I would very much like to have something I can run in order to test whether my changes to LuaJIT are vaguely sensible. Thus far, the LuaJIT-test-cleanup repo has contained some very useful raw material, but efforts have somewhat stalled. As such, I would like to shake things up and attempt to inject some life:

Some interesting features of my proposed runner are:

Wiladams commented 8 years ago

I'd say go for it.

Having that specific test runner in 'experimental' has exactly served the intended purpose. �It was not intrusive, and it's easy to dispose of.

I would also get rid of the "everything must be a pull request". �I really don't think that's necessary. �I have write permissions to the repo, so creating pull requests is superfluous, considering none of the history will be preserved in the end, and it just slows down the process (demotivating for me).

Party on dude! �Start pushing, or pull requesting!

Date: Sun, 3 Apr 2016 14:47:07 -0700 From: notifications@github.com To: LuaJIT-test-cleanup@noreply.github.com Subject: [LuaJIT/LuaJIT-test-cleanup] Shake things up a bit around here (#7)

I would very much like to have something I can run in order to test whether my changes to LuaJIT are vaguely sensible. Thus far, the LuaJIT-test-cleanup repo has contained some very useful raw material, but efforts have somewhat stalled. As such, I would like to shake things up and attempt to inject some life:

  • Do away with the idea of being conservative and putting things under the experimental directory to start with. This whole repo is going to undergo enough flux, and is sufficiently experimental anyway, as to not need an experimental directory.
  • Throw away @ladchttps://github.com/ladc's experimental test runner. On the one hand, I'm sorry. On the other, I feel that it didn't function well in a number of areas:
  • It required lfs, which despite being a libary I like, is a dependency I'd really rather not have.
  • Its codestrings weren't constructed with a view of maintaining line numbers, making it more difficult than neccessary to interpret backtraces.
  • Even in verbose mode, it wouldn't print the currently executing test until after the test had completed. This is of no use when tests fail due to segfaults or C assertion failures (these being common failure scenarios when making heavy changes to LuaJIT internals).
  • Add a new test runner, and convert 250+ tests to the format it expects.

Some interesting features of my proposed runner are:

  • The --shuffle=SEED command line option to vary the order of tests.
  • The --slice=INDEX/NUM_SHARDS command line option for parallelising things. The idea here is to have a higher-level makefile specify say luajit test.lua --slice=1/8 through luajit test.lua --slice=8/8 in 8 separate targets, and then have a merging target afterwards, at which point make -jN for N in [1, 8] will run the test suite using N threads.
  • Tags with numeric values rather than just true/false.
  • index files, to avoid an lfs (or similar) dependency, to provide deterministic and controllable directory iteration order, and to allow for file/directory level tags/other-metadata.
  • A README.md file.

You can view, comment on, or merge this pull request online at:

https://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7

Commit Summary

  • Shake things up.

File Changes

  • D experimental/test.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-0 (86)
  • D experimental/test/libs_ext/table.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-1 (37)
  • D experimental/tester.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-2 (274)
  • A test/README.mdhttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-3 (110)
  • R test/bc/constov.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-4 (35)
  • A test/bc/indexhttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-5 (1)
  • A test/common/expect_error.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-6 (16)
  • A test/common/test_runner_canary.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-7 (1)
  • A test/computations.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-8 (113)
  • A test/indexhttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-9 (6)
  • A test/lang/andor.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-10 (61)
  • A test/lang/assignment.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-11 (46)
  • A test/lang/compare.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-12 (249)
  • R test/lang/concat.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-13 (214)
  • A test/lang/constant/indexhttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-14 (1)
  • A test/lang/constant/number.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-15 (12)
  • A test/lang/coroutine.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-16 (8)
  • A test/lang/for.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-17 (45)
  • A test/lang/gc.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-18 (30)
  • A test/lang/indexhttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-19 (12)
  • A test/lang/modulo.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-20 (46)
  • A test/lang/self.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-21 (19)
  • A test/lang/tail_recursion.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-22 (20)
  • A test/lang/upvalue/closure.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-23 (84)
  • A test/lang/upvalue/indexhttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-24 (1)
  • A test/lib/base/assert.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-25 (33)
  • A test/lib/base/error.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-26 (43)
  • A test/lib/base/getfenv.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-27 (13)
  • A test/lib/base/indexhttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-28 (5)
  • R test/lib/base/select.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-29 (204)
  • R test/lib/base/xpcall_jit.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-30 (169)
  • A test/lib/bit.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-31 (98)
  • A test/lib/contents.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-32 (151)
  • A test/lib/coroutine/indexhttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-33 (1)
  • A test/lib/coroutine/yield.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-34 (109)
  • R test/lib/ffi/ffi_arith_ptr.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-35 (0)
  • R test/lib/ffi/ffi_bit64.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-36 (0)
  • R test/lib/ffi/ffi_bitfield.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-37 (0)
  • R test/lib/ffi/ffi_call.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-38 (0)
  • R test/lib/ffi/ffi_callback.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-39 (0)
  • R test/lib/ffi/ffi_const.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-40 (0)
  • R test/lib/ffi/ffi_convert.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-41 (0)
  • R test/lib/ffi/ffi_copy_fill.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-42 (0)
  • R test/lib/ffi/ffi_enum.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-43 (0)
  • R test/lib/ffi/ffi_err.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-44 (0)
  • R test/lib/ffi/ffi_gcstep_recursive.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-45 (0)
  • R test/lib/ffi/ffi_istype.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-46 (0)
  • R test/lib/ffi/ffi_jit_arith.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-47 (0)
  • R test/lib/ffi/ffi_jit_array.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-48 (0)
  • R test/lib/ffi/ffi_jit_call.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-49 (0)
  • R test/lib/ffi/ffi_jit_complex.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-50 (0)
  • R test/lib/ffi/ffi_jit_conv.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-51 (0)
  • R test/lib/ffi/ffi_jit_misc.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-52 (0)
  • R test/lib/ffi/ffi_jit_struct.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-53 (0)
  • R test/lib/ffi/ffi_lex_number.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-54 (0)
  • R test/lib/ffi/ffi_meta_tostring.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-55 (0)
  • R test/lib/ffi/ffi_metatype.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-56 (0)
  • R test/lib/ffi/ffi_new.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-57 (0)
  • R test/lib/ffi/ffi_nosink.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-58 (0)
  • R test/lib/ffi/ffi_parse_array.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-59 (0)
  • R test/lib/ffi/ffi_parse_basic.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-60 (0)
  • R test/lib/ffi/ffi_parse_cdef.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-61 (0)
  • R test/lib/ffi/ffi_parse_struct.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-62 (0)
  • R test/lib/ffi/ffi_redir.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-63 (0)
  • R test/lib/ffi/ffi_sink.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-64 (0)
  • R test/lib/ffi/ffi_tabov.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-65 (0)
  • R test/lib/ffi/ffi_type_punning.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-66 (0)
  • A test/lib/indexhttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-67 (8)
  • A test/lib/math/abs.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-68 (16)
  • A test/lib/math/constants.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-69 (8)
  • A test/lib/math/indexhttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-70 (3)
  • A test/lib/math/random.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-71 (47)
  • R test/lib/string/byte.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-72 (182)
  • A test/lib/string/char.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-73 (29)
  • R test/lib/string/dump.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-74 (60)
  • A test/lib/string/format/indexhttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-75 (1)
  • A test/lib/string/format/num.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-76 (184)
  • A test/lib/string/indexhttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-77 (9)
  • A test/lib/string/len.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-78 (14)
  • A test/lib/string/lower_upper.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-79 (16)
  • A test/lib/string/metatable.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-80 (3)
  • A test/lib/string/rep.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-81 (13)
  • A test/lib/string/sub.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-82 (189)
  • A test/lib/table/concat.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-83 (55)
  • A test/lib/table/indexhttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-84 (6)
  • R test/lib/table/insert.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-85 (7)
  • R test/lib/table/misc.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-86 (60)
  • A test/lib/table/new.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-87 (11)
  • A test/lib/table/pack.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-88 (7)
  • R test/lib/table/remove.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-89 (11)
  • A test/lib/table/sort.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-90 (27)
  • D test/misc/ack.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-91 (12)
  • D test/misc/ack_notail.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-92 (12)
  • D test/misc/argcheck.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-93 (40)
  • D test/misc/assign_tset_prevnil.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-94 (11)
  • D test/misc/assign_tset_tmp.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-95 (5)
  • D test/misc/bit_op.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-96 (90)
  • D test/misc/compare.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-97 (232)
  • D test/misc/exit_growstack.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-98 (24)
  • D test/misc/exit_jfuncf.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-99 (30)
  • D test/misc/fac.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-100 (13)
  • D test/misc/fastfib.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-101 (26)
  • D test/misc/fib.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-102 (11)
  • D test/misc/fuse.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-103 (8)
  • D test/misc/fwd_hrefk_rollback.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-104 (34)
  • D test/misc/libfuncs.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-105 (63)
  • D test/misc/math_random.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-106 (21)
  • D test/misc/modulo.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-107 (42)
  • D test/misc/nsieve.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-108 (20)
  • D test/misc/parse_andor.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-109 (58)
  • D test/misc/recsum.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-110 (11)
  • D test/misc/recsump.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-111 (11)
  • D test/misc/self.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-112 (18)
  • D test/misc/snap_gcexit.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-113 (16)
  • D test/misc/snap_top.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-114 (16)
  • D test/misc/snap_top2.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-115 (15)
  • D test/misc/sort.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-116 (23)
  • D test/misc/string_char.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-117 (28)
  • D test/misc/string_sub.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-118 (60)
  • D test/misc/string_sub_opt.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-119 (109)
  • D test/misc/tak.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-120 (12)
  • R test/opt/dse/array.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-121 (397)
  • R test/opt/dse/field.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-122 (147)
  • A test/opt/dse/indexhttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-123 (2)
  • A test/opt/fold/indexhttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-124 (1)
  • R test/opt/fold/kfold.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-125 (162)
  • A test/opt/fuse.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-126 (5)
  • A test/opt/fwd/hrefk_rollback.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-127 (32)
  • A test/opt/fwd/indexhttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-128 (3)
  • R test/opt/fwd/tnew_tdup.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-129 (147)
  • R test/opt/fwd/upval.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-130 (107)
  • A test/opt/indexhttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-131 (6)
  • A test/opt/loop/indexhttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-132 (1)
  • A test/opt/loop/unroll.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-133 (32)
  • R test/opt/sink/alloc.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-134 (266)
  • A test/opt/sink/indexhttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-135 (2)
  • R test/opt/sink/nosink.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-136 (233)
  • R test/src/cpptest.cpphttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-137 (0)
  • R test/src/ctest.chttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-138 (0)
  • A test/test.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-139 (409)
  • R test/trace/exit_frame.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-140 (159)
  • A test/trace/exit_growstack.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-141 (28)
  • A test/trace/exit_jfuncf.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-142 (30)
  • A test/trace/indexhttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-143 (5)
  • A test/trace/snap.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-144 (47)
  • R test/trace/stitch.luahttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7/files#diff-145 (40)

Patch Links:

� You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHubhttps://github.com/LuaJIT/LuaJIT-test-cleanup/pull/7