Record count changes when a different `pageSize` param value is used

[nickdos]

I'm only seeing this on the test server (not prod).

Compare these 2 requests:

https://biocache-ws-test.ala.org.au/ws/occurrences/search?q=occurrence_date%3A%5B2020-07-20T00%3A00%3A00Z%20TO%202020-07-30T00%3A00%3A00Z%5D&fq=state%3A%22Australian%20Capital%20Territory%22&qualityProfile=ALA&pageSize=20 results in

"totalRecords": 191

vs

https://biocache-ws-test.ala.org.au/ws/occurrences/search?q=occurrence_date%3A%5B2020-07-20T00%3A00%3A00Z%20TO%202020-07-30T00%3A00%3A00Z%5D&fq=state%3A%22Australian%20Capital%20Territory%22&qualityProfile=ALA&pageSize=10 results in

"totalRecords": 201

The only difference is one is using pageSize of 20 and the other is using a value of 10. However the number of records returned is different by 10 records.

• [ ] investigate and fix the issue of totalRecords changing with a change in pageSize value

[nickdos]

Changing pageSize to 30 results in 186 records returned.

[qifeng-bai]

By giving pageSize = 40, return size of records 235 35 : 235 30 : 235 25 : 239 20 : 244 10 : 254 5 : 259 1 : 263

And I tried on execute query on SOLR: SSH tunnel to : nci3-solr-3 ssh nci3-solr-3.ala -L 8983:localhost:8983

page size = 10, return 254

Page size = 20, return 244

[nickdos]

So seems like a SOLR issue - does prod do the same thing?

[qifeng-bai]

@nickdos It only happens on solr tests. Prod works fine

[nickdos]

We need to determine if it is broken due to a configuration issue or some code/schema change... Maybe we should create a new instance of biocache-ws-test and a SOLR instance... to rule that out?

[qifeng-bai]

I ran tests on collection of /solr/#/biocache-2021-09-03-11-07/ on nci3-solr-3 , it works OK.

@patkyn and I tested the collections on databox, they works correctly

It seems an issue with some collections on nci3-solr-3 and databox is fine.

[nickdos]

I think we should generate a new SOLR collection on nci test site - either by taking a copy of prod or data-box.

[adam-collins]

Test and prod look ok.