Hotware / Hibernate-Search-GenericJPA

Hibernate-Search with the JPA provider you want
http://hotware.github.io/Hibernate-Search-GenericJPA/
GNU Lesser General Public License v2.1
5 stars 3 forks source link

optimize JPAUpdateSource performance #93

Closed s4ke closed 8 years ago

s4ke commented 8 years ago

only select the most recent update event if that's possible.

s4ke commented 8 years ago

Such a query should do the job:

SELECT * FROM author_bookhsearchupdates t1 INNER JOIN ( SELECT max(t2.updateidhsearch) updateid, t2.authorFkfk, t2.bookFkfk FROM author_bookhsearchupdates t2 GROUP BY t2.authorFkfk, t2.bookFkfk ) t3 on t1.authorFkfk = t3.authorFkfk AND t1.bookFkfk = t3.bookFkfk AND t1.updateidhsearch = t3.updateid;

s4ke commented 8 years ago

when deleting the updates, we have to make sure that we delete all the updates we already handled. this can be done with something along the lines of

DELETE FROM author_bookhsearchupdates WHERE updateidhsearch < id_of_last_update

s4ke commented 8 years ago

This will reduce the database/indexing load heavily for incremental changes.

s4ke commented 8 years ago

deleting implemented.