Fix example test function

[cyblue9]

I tried to test using make test. As result, example/ tests were timeout. (tests/ are OK)

 6/41 Test  #6: tests/core/generics.xtm ..................................   Passed   21.81 sec
      Start  7: tests/external/fft.xtm
 7/41 Test  #7: tests/external/fft.xtm ...................................   Passed   14.15 sec
      Start  8: examples/core/audio_101.xtm
 8/41 Test  #8: examples/core/audio_101.xtm ..............................***Timeout 300.04 sec
      Start  9: examples/core/fmsynth.xtm
 9/41 Test  #9: examples/core/fmsynth.xtm ................................***Timeout 300.05 sec
      Start 10: examples/core/mtaudio.xtm
10/41 Test #10: examples/core/mtaudio.xtm ................................***Timeout 300.04 sec

Because,sys:load-then-quit function doesn't end until Timeout that set in CMakeLists.txt(300 sec) to use callback function.

So, I delete callback function in sys:load-then-quit. OK?

Please review!

[benswift]

Hey @cyblue9 the "load-then-quit" thing was designed so that certain example files could be used as integration tests; which includes letting them run for a while before quitting. Obviously it's not as good as actually having a human listen to the sound output to verify that it's correct, but it's better than nothing.

So removing the "delay before quit" thing will make it harder to use those example files as tests in this way.

They don't actually work that well, anyway - because depeding on what's going on with the dsp callback (quit 0) doesn't always exit cleanly anyway. So those "examples as tests" aren't currently used in the CI test job. But that's what the 'timeout then quit' thing is there for.

[cyblue9]

Hi @benswift !

Thank you for reply.

hum, I understood its. But, I don't think it's appropriate for a test that is passing to be Timeout in make test. How can we solve this problem?

[benswift]

Ah, ok, I see. So the test do pass if the test function calls (quit 0) directly at the end of the function body, but they don't pass if the (quit 0) happens in a callback (after a delay).

I suspect it could be to do with the fact that the tear-down of the DSP signal chain doesn't happen cleanly with (quit 0) (in fact, we don't really ever clean up the audio device stuff---mostly I just kick it in the head with SIGINT, which Extempore doesn't handle in any special way).

[cyblue9]

All right!

I'm sorry, I don't understand this mean...😥

I suspect it could be to do with the fact that the tear-down of the DSP signal chain doesn't happen cleanly with (quit 0) (in fact, we don't really ever clean up the audio device stuff---mostly I just kick it in the head with SIGINT, which Extempore doesn't handle in any special way).

For example,examples/core/nbody_lang_shootout.xtm is pass. Because, (quit 0) exists in last line. Adding (quit 0) to the last line of each example is not a solution, is it?

[benswift]

For example,examples/core/nbody_lang_shootout.xtm is pass. Because, (quit 0) exists in last line.

That's kindof a special case---it's designed for benchmarking, so making it quit once it's done means that you can run it with time at the CLI.

Adding (quit 0) to the last line of each example is not a solution, is it?

In general, no. Most of the examples do something which designed to be listened to, and often new users will just highlight the whole file and hit "eval", so if there was a quit at the end then they wouldn't hear the music the example is supposed to make.

Probably what we need to do is re-think which examples we actually list as tests, because I agree that it's annoying that make test isn't actually a reliable indicator of what's working and what's not (and you shouldn't have to file a PR for me to answer you with the "secret knowledge" to find out :grin:).