Feature request: Support distinct option for de-duplication

algolia / algoliasearch-django

Seamless integration of Algolia into your Django project.

https://www.algolia.com

MIT License

172 stars 65 forks source link

Feature request: Support distinct option for de-duplication #271

Open m-vdb opened 5 years ago

m-vdb commented 5 years ago

Description

Documentation: https://www.algolia.com/doc/guides/ranking/distinct/#distinct-for-de-duplication

I am hitting the 10kb limit for records and I need to be able to support de-duplication. Here is my use case:

I have one model that contains a list of locations
for some models, this list contains a lot of locations (some thousands)
when indexing my model, I want to push 1 record for each location in the list: this will tremendously decrease record size while at the same time enhance the search experience

Are you willing to accept PRs for this feature? I can draft a spec here and get on with it 💪 Cheers!

m-vdb commented 5 years ago

no one?

clemfromspace commented 5 years ago

Hey @m-vdb,

Sorry for the late answer, we will definitely accept a PR for this case.

Had a quick look at your fork, the global logic looks good so far :)

Happy to review your PR when it's ready!

m-vdb commented 5 years ago

thanks @clemfromspace! I'll be adding tests too. Will let you know when it's ready

m-vdb commented 5 years ago

hey @clemfromspace I finally got the time to finish my work on the PR. I have coded integration tests to validate that everything works. IMO, what's missing:

validate my API: is it clear enough for other developers (method names, comments, etc...)?
documentation in the README: I can do it once you validate the API
real life tests for the AlgoliaIndexBatch: I haven't tested it yet with real data. In my case, I have records that will "duplicate" to thousands of records. It's next on my list.
compat code: I had to keep compat code for Django 1.7, I don't know if you'll agree with that. Ideally, it'd be better if algoliasearch-django dropped support for 1.7, but that's up to you.

You can see my new pull request on this repo here: https://github.com/algolia/algoliasearch-django/pull/274

yhoiseth commented 4 years ago

What's the status on this?

soulshake commented 4 years ago

I would like to use this functionality. Is there any plan to merge #274?

francois-travais commented 2 years ago

I would also need this one. Is there any plan to do something about it?