search

hbz / mabxml-elasticsearch

Raw hbz union catalog data exposed via a web API
http://lobid.org/hbz01
3 stars 1 forks source link

Missing updates #57

Closed dr0i closed 3 years ago

dr0i commented 3 years ago

Originating from https://github.com/hbz/lobid-resources/issues/1227: some incremental updates cannot be indexed, beginning with /data/DE-605/mabxml/update//DE-605-aleph-update-marcxchange-20210425-20210426.tar.gz. Unfortunately the logs don't give a clue besides the failing with status code 500.

Suspecting insufficient memory because the process is given only 1GB of RAM while the incremental file is ~3GB. Will restart the process with temporarily more memory.

dr0i commented 3 years ago

Increased java xmx to 8GB, without success though :(

dr0i commented 3 years ago

Ah, I got it: the cause is also a race condition, resulting in copying only partly finished archives.

dr0i commented 3 years ago

It's working now, see e.g. https://lobid.org/hbz01/HT020901818. Closing.