mock_uss interaction logging endpoint requires an excessive amount of time

[BenjaminPelletier]

Describe the bug When querying mock_uss's /interuss_logging/logs, mock_uss takes an excessive amount of time to return results (more than 5 seconds). Because uss_qualifier's timeout is 5 seconds, this causes scenarios relying on this reporting to fail.

To reproduce

• Deploy mock_uss in a real testing environment
• Run F3548 suite a number of times
• Observe that queries to the logs endpoint take a long time; see screenshots below from a mock_uss instance deployed with GKE

Difference from expected behavior The mock_uss logs endpoint should respond quickly; seems like sub-1s should be easily achievable.

Possible solution Include filter criteria in file names so that content doesn't need to be loaded. Consider using tracer to obtain this information instead.

Screenshots

System on which behavior was encountered Google Kubernetes Engine with one e2-standard-4 node

Codebase information /status output:

Mock USS ok interuss/monitoring/v0.6.0-abc0bac74; hosting versioning, interaction_logging, flight_planning, scdsc

Notes from the 2024-08-13 Community Meeting

We want to offer endpoints that are robust and precise: the ideal fix should allow to precisely retrieve events that occured at, or later, than a requested time.

At the same time, we assume that clients will properly deal with synchronizing their local clocks and we should not need to include mitigations for clock skew. (If clock skew should be an issue, this can be handled on the qualifier side)

[Shastick]

Taking a look at this, as I need to get better acquainted with the mock_uss for solving #693

Notes

We seem to be saving the interactions using the current time in the file name.

When looking up interactions, we iterate over every logged interaction file

A simple change would be to save the time of the interaction in the filename, using the timestamp of the request from the Query wrapped in the interaction, allowing us to filter out the relevant interactions without the need to open files and parse their content.

Observations

We seem to log interactions via a query hook and a @webapp.after_request annotation, which might means we might try to log interactions at the same time. This is relevant for the interaction file naming.

[Shastick]

A first proposal for improving things is suggested in https://github.com/interuss/monitoring/pull/750

This is not guaranteed to fix things as, depending on the requested time from which we might still need to iterate over many files.

If the performance is still not giving satisfaction, we could consider returning the interactions without re-parsing them from the files.

[Shastick]

@BenjaminPelletier did you get a chance to observe the performance of the new version of the mock_uss (containing #750?) If performance has been improved we can close this issue.