ska-sa / spead2

Library for the Streaming Protocol for Exchange of Astronomical Data (SPEAD)
http://spead2.readthedocs.io/en/latest/
GNU Lesser General Public License v3.0
23 stars 14 forks source link

Add chunk stream groups #219

Closed bmerry closed 1 year ago

bmerry commented 1 year ago

This allows multiple streams to work on assembling a shared chunk, giving better scalability provided the incoming data can be distributed across streams. A good starting point for review is doc/recv-chunk-group.rst.

This is quite a large PR, because it also incorporates a change in the way streams are shut down.

I'm not totally happy with the amount of testing, but I'm starting the ball rolling, and I can always add more tests before release.