jdries
commented
4 months ago I had a look at metrics and spark UI. There was a slow period in april, but this went away somehow. Depending on the time of day, there's also still requests that are faster than 4 seconds, so the average doesn't show a very consistent slowdown. The metrics also no longer show a period when it was consistently better. A screenshot would have been a good idea, or else storing metrics for a bit longer than only 3 months... Note that these metrics show the average over all queries, which can be a very mixed group. A slow query can for instance happen when a user simply selects a large area or large time range.
So right now, it's hard to tell if there's an issue. For users that are on the old TS api, we first need to ask them to migrate to openEO. For users that think openEO timeseries is still to slow, they should give us their specific case, so we can investigate and optimize.
The average timeseries requests appear to have gone from 4 seconds to 8 seconds (Grafana) with outliers that take a few minutes to complete a simple request.
These slow request times are impacting services such as cropsar. Where we had to increase the alerting threshold because 90% of a cropSAR request is spent waiting on a response from our timeseries service.