search

elastic / connectors

Official Elastic connectors for third-party data sources
https://www.elastic.co/guide/en/elasticsearch/reference/master/es-connectors.html
Other
18 stars 137 forks source link

Flaky test: tests/test_service_cli.py::test_shutdown_called_on_shutdown_signal[Signals.SIGINT] #2517

Open artem-shelkovnikov opened 6 months ago

artem-shelkovnikov commented 6 months ago

One unit test is flaky and from time to time takes too much time to run, causing CI to terminate the process: bin/pytest tests/test_service_cli.py::test_shutdown_called_on_shutdown_signal

artem-shelkovnikov commented 6 months ago

Tried looking into it and it's really weird, feels like it's an interrupt that happens in a wrong moment.

Testing it a bit locally showed that the test does not interrupt because some services (in my case it was job cleanup service) did not exit.

I think it happens because SIGINT/SIGTERM happens in between service startup - e.g. Job Scheduling/Job Execution Services has just started, next Job Cleanup Service will attempt to start, but interrupt happens before and service just starts.

☝️ This info is just a theory, I was unable to prove or discard it.

seanstory commented 5 months ago

I wonder if this is related to https://github.com/elastic/connectors/issues/2568

artem-shelkovnikov commented 4 months ago

@seanstory I have a feeling it's the same issue.

It feels like CTRL+C in some particular place of code does not correctly trigger the cleanup and after cleanup was started some couroutines were still executed.