Would switching results to redux allow incremental loading?

[dosumis]

We have scaling issues with some results pages. This is something we predicted with our query approach from the beginning and we initially proposed lazy loading as a solution (An initial query finds the first n results, a second query finds the rest and updates the table). As I understand it, this is not possible with the current results table, but might be if we switch to redux. I'd like to investigate whether this is possible, as scaling issues are likely to get worse in the near future.

Potential challenges:

• results ordering is difficult to achieve with partial queries, so table may initially need to be un-ordered.
• ordering/refining will need to be disabled until the query completes and this needs to be clearly indicated to users.

Initial step: investigate whether switching to redux would support this.

[ddelpiano]

this has been already discussed in the previous months during sprint call, just making a recap here in the card. In order to have a lazy loading mechanism the biggest chunk of work will be on the backend side since the results will need to be chunked and served to the frontend using pagination or anyway serving the results in different and smaller steps. This will impact mainly the datasources bundle since this will have to handle the serving of the results as described above but we need to be sure also of which other stacks we are working with and what do they support, neo4j for instance does support limit and this can be used to implement a pagination mechanism, not sure about the other datasources and some researches will be needed for them. Thx