search

simvue-io / integrations

A repository to store the integrations (or Connectors) of the Simvue Client for different use cases
Apache License 2.0
0 stars 0 forks source link

CTRL log data sometimes not sent to events / metrics #21

Open wk9874 opened 1 month ago

wk9874 commented 1 month ago

Sometimes the data kept inside the CTRL log CSV file doesnt get sent to the events and/or metadata on the server. This appears to be a timing issue with two parts:

  1. The multiparser tail CSV parser seems to not send all data added to the CSV file to the callback. Adding a print inside the for loop in the CSV parser shows all lines are read, but then adding a print into the callback shows only some are parsed and sent there
  2. If two threads try to add metadata to a run at the same time, one piece will be discarded by the server

See the failing test test_fds_ctrl_parser

wk9874 commented 1 month ago

Linked to #18