Negative number of missing events logged - Githubissues

octue / octue-sdk-python

The python SDK for @Octue services and digital twins.

https://octue.com

Other

10 stars 4 forks source link

Negative number of missing events logged #661

Closed cortadocodes closed 4 months ago

cortadocodes commented 5 months ago

Bug report

What is the current behavior?

A negative number of missing events are being logged sometimes. I get the following warning logged:

[WARNING | octue.cloud.events.handler] [analysis-1588e25e-b51c-4af4-bc62-fbc175d31139] <Service('gulfwindtech/dlc-service:add-real-dlc-ser')>: -108 consecutive events missing for question '84cc924e-13e5-4769-9eb9-9c1bed405c53' after 10s - skipping to next earliest waiting event (event 3).

See here for the logs this came from.

What is the expected behavior?

This warning should only be triggered when there are missing events. If there are missing events, this should always be a positive number.

Your environment

Running in Cloud Run
octue==0.56.0 (currently the improve-ask-multiple branch)
I'm using Child.ask_multiple, so this might be due to some non-thread-safe behaviour

Proposed Solution

Investigate whether:

There are actually missing events when this occurs
Whether this only occurs when using Child.ask_multiple
Whether there's some thread leakage going on

cortadocodes commented 4 months ago

This is no longer relevant as I've removed event ordering from the event handler in the upcoming release