search

tahseen / amazon-cloudsearch-client-java

Amazon CloudSearch Client for Document Service and Searching
18 stars 22 forks source link

Batch limits #6

Closed smillerk2 closed 7 years ago

smillerk2 commented 9 years ago

Is there any plan to handle document add requests that would surpass the 5Mb batch upload limit or single document size? http://docs.aws.amazon.com/cloudsearch/latest/developerguide/limits.html

tahseen commented 9 years ago

Probably best way to get an answer to post your question here https://forums.aws.amazon.com/forum.jspa?forumID=137

smillerk2 commented 9 years ago

Sorry, to be clear, it would be great if your library could internally deal with issuing multiple batch requests if the serialized content exceeds the AWS limit. Currently, I could add thousands of large items to your list and this would fail when received by AWS.

tahseen commented 9 years ago

Ah I see. I will reopen the issue.

I am not sure yet how to efficiently implement this at the moment.

smillerk2 commented 9 years ago

Thanks for looking into it. My Java skills are non existent but here is an interesting concept that was done in .NET : https://github.com/martin-magakian/Amazing-Cloud-Search/blob/master/AmazingCloudSearch/CloudSearch.cs