search

kermitt2 / biblio-glutton

A high performance bibliographic information service: https://biblio-glutton.readthedocs.io
117 stars 15 forks source link

Feature request/Question: Keep data up to date, incremental appending #49

Closed karatekaneen closed 2 years ago

karatekaneen commented 3 years ago

First of all, love the project and really appreciate the work that you are doing!

We have an issue where we want to keep the data as fresh as possible and update it often. By the standard way that are described in the documentation with the bulk datasets this can be done by appending to the huge files and then re-running the indexing. So my question is if there is any way currently to simply add new items (crossref, pubmed etc) or if you have any pointers of where to start if I was to add that API for you?

Best regards, Robin

kermitt2 commented 2 years ago

Implemented with #66 and version 0.2. Daily update is set at a time given in the config file (default 03:00 local server time).

karatekaneen commented 2 years ago

I've actually been running the branch from #66 in our prod environment for a little while after I worked out all the kinks mentioned in the PR. Amazing work!