nthallen / monarch

Monarch Data Acquisition System
0 stars 1 forks source link

tm_generator bfr backup #100

Closed nthallen closed 4 years ago

nthallen commented 4 years ago

rdr is failing when the read data comes in faster than bfr can dispose of it. We need to be able to throttle the input. This was handled via separate threads in QNX, but I am trying to avoid that.

I propose instead to provide a fallback output buffer in tm_gen_bfr. If an iwritev to bfr does not transfer all its data immediately, the data can be written into the backup buffer, and a bfr_blocked() method called. When the data is ultimately written, bfr_unblocked() will be called.

The size of the backup buffer should be a little bit larger than the tm_generator's source data. Specifically, I need to able to continue writing to tm_gen_bfr throughout an entire call to tm_rcvr::process_message(). For rdr, this could be an entire data file (4096*3 max size). The internal TM protocol is a bit more verbose than the lgr data files, so we need to allow for additional space for header information. In practice, this should really amount to no more than one or two headers per file, particularly since the initial timestamp in each file is redundant and not transmitted.

I should record a max_bytes_per_message to find out.

nthallen commented 4 years ago

Fixed in c47eb3ebb1b897b3203f6a0c1170755903802e53