search

tklab-tud / uscxml

SCXML interpreter and transformer/compiler written in C/C++ with bindings to Java, C#, Python and Lua
Other
106 stars 54 forks source link

add -o2 to scxmlcc test #181

Open jp-embedded opened 6 years ago

jp-embedded commented 6 years ago

scxmlcc is designed to make the compiler able optimize away empty actions, etc. For this to work, optimizations must be enabled, so the scxmlcc test program really should be compiled with -O2 when testing performance.

sradomski commented 6 years ago

When I added compiler optimizations back then, the time required for the compilation increased to an unacceptable level. As you removed the timeout invocation, I am curious if you have actually run the benchmarks with -O2?

jp-embedded commented 6 years ago

Yes, I have run the benchmarks

Yes, it is quite challenging for scmlcc to handle this many parallel children. I just thought a long compilation time was better than no result.

But some of the tests takes quite some time for uscxml also. For example the fast LCCA.256 takes 30s to initialize and the fast Transitions.256 takes 56s. (on my machine) - see log below

I have now put the timeout back on and increased it to 2 minutes. This allows more of the tests to complete for both uscxml and scxmlcc.

What do you think of that?

== Running LCCA.256 from /home/jp/work/uscxml/test/benchmarks/LCCA.256.scxml ==== with USCXML [Info] Registering at unstarted HTTP Server [Info] Registering at unstarted HTTP Server 30426, 6 30426, 6 30426, 6 30426, 5 30426, 6 30426, 6 30426, 6 30426, 6 30426, 6 [Info] Registering at unstarted HTTP Server [Info] Registering at unstarted HTTP Server 779, 26 779, 25 779, 25 779, 25 779, 25 779, 25 779, 25 779, 25 779, 25 == Running Transitions.256 from /home/jp/work/uscxml/test/benchmarks/Transitions.256.scxml ==== with USCXML [Info] Registering at unstarted HTTP Server [Info] Registering at unstarted HTTP Server 56104, 551 56104, 584 56104, 519 56104, 521 56104, 506 56104, 487 56104, 441 56104, 472 56104, 484 [Info] Registering at unstarted HTTP Server [Info] Registering at unstarted HTTP Server 570, 656 570, 841 570, 836 570, 845 570, 831 570, 842 570, 826 570, 858 570, 677

sradomski commented 6 years ago

The fast implementation precalculates all predicates, whereas the large interpreter calculates and caches them on the fly. I’ll merge and update the graphs as soon as I have some time on my hands. I’ll leave this open as a reminder to myself.