Reception events not consumed in order of receipt (file based in take)

[FreddieMatherSmartDCSIT]

Hi munin team,

An issue with the file based input that became apparent during testing - we are aware of Kafka message broker which I'm guessing will likely have fixed this issue already but raising for visibility.

Reception files are not consumed in order of when they are received. It was noted that once the rate of events was too high events sent towards the end of the test (50 minutes in) were being received when event sent before that had still not been picked up by AEReception. This was leading to jobs not being fully processed after the time when the input rate became too high.

Steps to reproduce:

• PV1.0.0
• Run 1 AEReception containers and at least 1 Verifier Container
• Send 500 events/s for 40 mins (sent to reception incoming folder)
• Job sequences are 33 events with 1 second interval between them
• Get timestamp of when each event id is sent to reception (event id timestamp pair)
• Get timestamp of all events received by reception (event id timestamp pair)
• Confirm there is a mismatch between the ordering of events with respect to time

[cortlandstarrett]

Closing this issue as Won't Fix. We use the file-based input only for small scale, local engineering development. In this environment, the ordering does not present a problem.

In the production configuration the message broker is keeping the order by design.