search

WormBase / pseudoace

Modelling the WormBase ACeDB database in datomic.
4 stars 3 forks source link

Gracefully handle interrupted import process #25

Closed mgrbyte closed 8 years ago

mgrbyte commented 8 years ago

When an import process fails (i.e import-logs) (e.g if the transactor dies) enable the process to be re-run and pick up from where it left off.

  1. Retrieve the timstamp T of the most recent transaction
  2. Filter input files <= T
  3. Guard the call-site in the import process to datomic.api.transact to skip the transaction if T is in the past.
mgrbyte commented 8 years ago

@a8wright highlighted that filtering of edn files by date off-by-one (need to apply clj-time.core/after? to the date before the latest-transacton-date.

mgrbyte commented 8 years ago

Fixed by #32