Determine Measurement Limit for new Data Explorer

influx-product-manager commented 2 years ago

Spike Description

In the old data explorer, there was a hard limit of 200 or 500 entries in the measurement list, depending on a feature flag setting.

We can assume this was done because of performance and the impact of gathering a large list and all the associated metadata.

With the performance improvements that have been made in both back-end and front-end and with the different way we are handing metaqueries, we should determine if this limit is still necessary, and if so - what limit is feasible.

It certainly seems like we could show more than 200 entries without it being problematic.

This spike is for doing some testing and analysis to determine what our options might be here.

Deliverable

Recommendation on what (if any) limit can be put on the measurement list in the new data explorer

influx-product-manager commented 2 years ago

Link to feature: https://influxdata.productboard.com/feature-board/planning/features/14285542

asalem1 commented 2 years ago

After doing some investigative work, the limit on the query doesn't appear to effect the performance of the dropdown to a certain point. When stress testing 10K measurements, the data was received and parsed in less than a second. However, performance issues started to crop up within the Dropdown component itself. Since React needs to keep track of each individual item in the dropdown, the longer the list, the more unperformant the dropdown becomes. All of that is to say that if we decide to remove the limit on the measurements dropdown, we should invest time to update the dropdown to only render a limited number of results that can then be paginated through endless scrolling or some other mechanism.

Prognosis

Removing the limits on the query does has no effect on the speed of fetching and parsing the data
Removing the limits on the query CAN effect the speed of the dropdown as the number of items increases, due to React's state management

Potential Solution

Update the Dropdown component in clockface to paginate results with endless scrolling so that only a subset of the items are ever displayed to the user and the component is still performant
In conjunction with the point above, we would want to audit and remove limits on data being sent from the API. More specifically to this issue, we would want to remove any limits being set on measurements for displaying in the UI

influxdata / ui