Run tests in parallel.

[cmumford]

Running one test for each physical CPU to improve test run performance.

[cmumford]

I couldn't build MicroPython on my system to verify this change. Also, I was worried that the test output might be intermixed for the parallel tests. I did the following, and they seem to be written to stdout in order:

find . -type d | xargs --max-args=1 --max-procs=8 ls -l

[cmumford]

Well the good news is that tests run twice as fast. 1m52s → 51s. However, I'm not sure the test output is grouped for each test. Maybe somebody more familiar with the test output can confirm. In the meantime I'll look to see if there is an existing Linux application to serve as a replacement for xargs.

[cmumford]

Verified that parallel does in fact produce the same ordered output as the single core xargs.

I would like to avoid adding another dependency (parallel) to downstream projects, but I think that doubling (or more) performance is probably worth it.

xargs in serial:

From https://github.com/lvgl/lv_binding_micropython/runs/3335118710:

Running lib/lv_bindings/tests/../examples/example3.py ...
lvgl obj 
lvgl btn 
lvgl label  text:">"
lvgl keyboard 
lvgl textarea  text:""
lvgl label  text:""
Running the SDL lvgl version

xargs in parallel:

Running lib/lv_bindings/tests/../examples/example3.py ...
lvgl obj 
lvgl obj 
lvgl btn 
lvgl chart 
lvgl label  text:">"
lvgl slider  value:"100"
lvgl keyboard 
Running the SDL lvgl version

parallel in parallel:

Running lib/lv_bindings/tests/../examples/example3.py ...
lvgl obj 
lvgl btn 
lvgl label  text:">"
lvgl keyboard 
lvgl textarea  text:""
lvgl label  text:""
Running the SDL lvgl version

[amirgon]

In case of an error, all tests should stop:

https://github.com/lvgl/lv_binding_micropython/blob/f4a6818a944da1691d1bc01766a356078d57ddfc/tests/run_test.py#L94

run_test.py uses exit code 255 to signal xargs that it should stop immediately.

I've skimmed through "parallel" docs and I couldn't find an indication that it behaves the same way, but it may be possible to use the --halt-on-error option to make it stop completely on the first error.

In any way, we should test both the "success" and "failure" cases when changing the test scripts.

I couldn't build MicroPython on my system to verify this change.

If for some reason you can't build lv_micropython locally - you can open it in GitPod instead. You'll be able to both build and run the unix port there.

[cmumford]

@amirgon you were correct - without --halt parallel will execute all tests. I added --halt-on-error now,fail=1 and parallel now stops on the first failure, and echos out the same stderr/stdout. echo $? reported 255 as the parallel return code.

I created the failure by editing lib/lv_bindings/lvgl/examples/layouts/grid/lv_example_grid_5.py and raising an exception.

Edit: run.sh executes in 1m51s with xargs and 21s with parallel on my i7-4790.

[amirgon]

Looks good! I vote for merging this.

I've merged https://github.com/lvgl/lv_micropython/pull/39. @embeddedt Any further comments? Could you merge https://github.com/lvgl/lvgl/pull/2510?

[embeddedt]

I've merged both PRs. Thank you!