WolfgangFahl / ConferenceCorpus

ScientificEventCorpus
Apache License 2.0
1 stars 2 forks source link

allow updating the database via webserver #24

Open WolfgangFahl opened 2 years ago

WolfgangFahl commented 2 years ago

e.g. running location fixers, ordinal and other proceedings parser parts or e.g gnd data range ..

Since these might be long running transactions use SSE to display progress and have some kind of transaction mechanism / version handling.

E.g. copy EventCorpus.db -> EventCorpus-2022-01-10_0936.db before starting task (e.g. location fix for a manager such as confref)

Followup problem is that sqlite will lock the database so the handling of the sqlite connection needs to be taken care of and some small "maintainance" statement might have to be set or there might be to different versions for public display and maintainance upgrade work.

WolfgangFahl commented 2 years ago

One idea is to use conferencercorpus.bitplan.com as the public version of the corpus and the rwth-i5 as the development/maintainance server. Thus it will be possible to trigger the update of the publich server nightly or on demand.