GLAM currently serves keyed probes by loading and processing all keys at once. This takes the back-end server an excessive amount of time, which in turn ends up triggering a timeout between the back-end server and the Load Balancer, causing an error on the application.
(Try this probe, for example)
In order to avoid this we should only fetch one key at a time per request and trigger a re-fetch whenever a new key is selected.
Since we do not know all the keys for a given probe beforehand, the challenge with this approach is to display the list of keys without loading all the data. I'll experiment with fetching keys only, then data for the first key and see how quickly it will be.
edugfilho
commented
1 week ago
GLAM currently serves keyed probes by loading and processing all keys at once. This takes the back-end server an excessive amount of time, which in turn ends up triggering a timeout between the back-end server and the Load Balancer, causing an error on the application. (Try this probe, for example)
In order to avoid this we should only fetch one key at a time per request and trigger a re-fetch whenever a new key is selected.
Since we do not know all the keys for a given probe beforehand, the challenge with this approach is to display the list of keys without loading all the data. I'll experiment with fetching keys only, then data for the first key and see how quickly it will be.