search

dplassgit / d2lang

D2 is a strongly-typed, statically-typed, (mostly) inferred-type compiled language.
MIT License
5 stars 1 forks source link

Streamline tests #310

Open dplassgit opened 6 months ago

dplassgit commented 6 months ago

Right now it takes about 8 minutes to run the full test suite (via bazel) on my laptop. I believe this is because of all the "variations" in tests - specifically in the NasmCodeGenerator{Byte,Double,Int,Long,String}Test files. I think the byte tests can be streamlined, since it's nearly the same logic as ints. Some of the long tests can be as well (though long constants are treated differently than int or byte constants.)

I'd love to get the test time down to about 4 minutes.

dplassgit commented 6 months ago

From clean, Nasm{Int,Long,Byte,String} in the trunk takes:

//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorArrayTest      PASSED in 57.3s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorBoolTest       PASSED in 68.6s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorBuiltinsTest   PASSED in 30.1s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorByteTest       PASSED in 104.5s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorDoubleTest     PASSED in 100.6s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorIntTest        PASSED in 103.6s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorLongTest       PASSED in 104.5s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorProcTest       PASSED in 30.0s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorStringTest     PASSED in 129.6s

Total: 726 seconds

dplassgit commented 6 months ago

After reducing/combining test cases (not committed yet)

//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorArrayTest      PASSED in 68.9s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorBoolTest       PASSED in 54.2s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorBuiltinsTest   PASSED in 30.5s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorByteTest       PASSED in 80.0s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorDoubleTest     PASSED in 63.6s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorIntTest        PASSED in 94.2s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorLongTest       PASSED in 86.7s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorProcTest       PASSED in 22.2s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorStringTest     PASSED in 126.4s

Total: 623 seconds... hm, sometimes it is much faster...

dplassgit commented 6 months ago

Remove redundant files from samples/ - there are way too many

dplassgit commented 5 months ago

In NasmCodeGeneratorTestBase, it compares interpreted to compiled, AND optimized vs not-optimized. It should be sufficient to test interpreted+not optimized to compiled+optimized. This will by definition reduce the runtime in half.

dplassgit commented 1 month ago

Now that issue #329 is fixed, by default it only ever compares optimized asm vs unoptimized interpreted. Runtimes are much faster now:

//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorArrayTest (cached) PASSED in 38.6s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorBoolTest (cached) PASSED in 55.1s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorBuiltinsTest (cached) PASSED in 31.4s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorByteTest (cached) PASSED in 54.8s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorDoubleTest (cached) PASSED in 45.0s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorIntTest (cached) PASSED in 75.0s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorLongTest (cached) PASSED in 61.4s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorProcTest (cached) PASSED in 15.7s
//test/com/plasstech/lang/d2/codegen/x64:NasmCodeGeneratorStringTest (cached) PASSED in 70.8s
dplassgit commented 3 weeks ago

In NasmCodeGeneratorTestBase, it compares interpreted to compiled, AND optimized vs not-optimized. It should be sufficient to test interpreted+not optimized to compiled+optimized. This will by definition reduce the runtime in half.

Unfortunately this is not a good idea. Compiled+optimized sometimes winds up with just a string print, avoiding much of the x64 codegen.

This issue might have to be completely reverted.