search

leekelleher / umbraco-contentment

Contentment for Umbraco - a state of happiness and satisfaction
https://marketplace.umbraco.com/package/umbraco.community.contentment
Mozilla Public License 2.0
158 stars 73 forks source link

Data Picker only returning 500 results #420

Open mastrup opened 1 month ago

mastrup commented 1 month ago

Which Contentment version are you using?

5.0.2

Which Umbraco version are you using? For example: 13.2.1 - don't just write v13

13.4.1

Bug summary

I have configured my datatype to use the [Contentment] Data Picker and as data source I have selected Examine Query. I have selected a custom index that currently holds 17429 items. My configuration looks like this: image

An item in the index looks somewhat like this: image

Whenever I use the datatype on a given document type it never returns more than 500 results. I have tried to search with my Lucene query (+__IndexType:damDefinition) directly on the Examine Management tab in the Settings section where I have verified that the query returns 17429 results. image

Steps to reproduce

  1. Create a datatype that uses the [Contentment] Data Picker and use Examine Query as data source.
  2. Select an index that holds more than 500 items and make sure that the Lucene query will return more than 500 results.
  3. Add the data type to a node to test the query.

Expected result / actual result

When using the data picker on a node it should return all of the results.

Do you have Umbraco ModelsBuilder enabled?

What browsers are you seeing the problem on?

Chrome

leekelleher commented 1 month ago

@mastrup Thank you for reporting this.

I suspect I know what's going on here... but first, a backstory... a while back I noticed that adoption of Data Picker was lower than I'd anticipated, I couldn't understand why... after chatting with a few people about it, it was the lack of supported data-sources, which were fully async with search and pagination. To do this across all the available Data List data-sources (34 of them) would have taken me quite a while, (and my focus should really be on Umbraco Bellissima migration), so I put an interim solution in-place.

In 0e8a95f6366ebfbad5e08ce5d234ca7e640a28f7, I added a Data List to Data Picker data-source bridge, so that Data Picker could work with any of the existing Data List data-sources. Which is fine, until it's not (like the scenario you've outlined).

With the Examine data-source isn't doing any async/pagination, it's purely doing a direct query on the index searcher, then the Data Picker bridging code is handling its own search/pagination on that dataset. e.g. it's faking it.

An interesting side-note here is, why is the Examine index searcher only returning a maximum of 500 results? (e.g. there's nothing in Contentment code that is setting that limit.)

So, what's the solution? It's either roll your own custom Data Picker data-source; or fix up the ExamineDataListSource.cs code to remove the bridging base class and add native support the async/search/pagination.

~I'll mark issue this as up-for-grabs,~ as my current free time of Contentment is focused on the Umbraco Bellissima migration (Contentment v6), and I'd love to have a fix like this as an community contribution. ❤️

Edit: Created a separate feature request with more explanation: #421

mastrup commented 1 month ago

@leekelleher Thanks for the quick response!

I actually started implementing my own Data Picker data-source and tried to inject the ExamineManager in order for me to search the index. That gave me the error Object reference not set to an instance of an object and I couldn't figure out how to solve it. I then discovered the option to use Examine Query and went with that until I discovered the issue.