DendraScience / issues

Any issues you have with the Dendra system, please log here.
1 stars 0 forks source link

Pseduo Binary Decoder, better handling of corrupt data #25

Open jscottsf opened 6 years ago

jscottsf commented 6 years ago

It's suspected that anomalous binary data (Core8) caused the Decoder's state machine to reach its max iterations. This, in-turn, created a choke-point for all Core8 decoding.

Fortunately, all data was still archived.

Restarting the DPE worker resolved the problem, and eventually the missing data gap was automatically filled.

Possible resolution would be to add an additional error hander to the Decoder class and set a buffer limit for the worker to ensure really-really bad data doesn't trip things up

jscottsf commented 6 years ago

This is likely not a bug with the Decoder. Seems that the issue is caused by an unforgiving "sub_options.ack_wait" setting for the DPE worker. In a constrained environment, the Decoding takes a bit longer since it's sharing the workload with other activities, hence, the ack_wait needs to be increased to prevent a race condition.

jscottsf commented 6 years ago

Before closing this issue, some followup issues should be opened: