ezmsg-org / ezmsg

Pure-Python DAG-based high-performance SHM-backed pub-sub and multi-processing pattern
https://ezmsg.readthedocs.io/en/latest/
MIT License
10 stars 5 forks source link

Enhancement: Introduce performance benchmarking into continuous integration #7

Open griffinmilsap opened 1 year ago

griffinmilsap commented 1 year ago

It would be lovely to get a textual or ideally graphical representation of performance testing tests/test_perf.py for various message sizes and pub-sub configurations including pub-sub in same process (TX_LOCAL), on same machine (TX_SHM), and on different machines (TX_TCP; can be forced with force_tcp in Publisher constructor so as to avoid a requirement for multiple machines in CI). Test should also vary message size (in powers of 2) from 8 bytes to ~2MB messages, and also evaluate fan-out (1, 2, 4, 8 subs connected to 1, 2, 4, 8 pubs). To get accurate numbers, the test-duration should probably be 10-30 sec.

Given this is probably a time-consuming test and performance regressions shouldn't necessarily be blocking for pull-requests (sometimes new features come at the expense of performance); this test should be configured as a separate GitHub Action.