search

Flowminder / FlowKit

FlowKit: Flowminder CDR analytics toolkit
https://flowminder.github.io/FlowKit/
Mozilla Public License 2.0
86 stars 21 forks source link

Remove file move ops from FlowETL #946

Closed greenape closed 5 years ago

greenape commented 5 years ago

At the moment, flowetl spends quite a lot of time moving files around. There's no great benefit to doing this, given their ingested or not state is tracked anyway.

Files should stay put.

josh-gree commented 5 years ago

What if you have an ssd and a normal disk? You want to do the read into the dB from the ssd and the long term storage on the normal disk. Need to at least move once.

On Tue, 18 Jun 2019, 13:06 Jonathan Gray <notifications@github.com wrote:

At the moment, flowetl spends quite a lot of time moving files around. There's no great benefit to doing this, given their ingested or not state is tracked anyway.

Files should stay put.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Flowminder/FlowKit/issues/946?email_source=notifications&email_token=ACDP2SY3G6IUPIJ45FGWWU3P3C6RRA5CNFSM4HY63X52YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4G2DOUKQ, or mute the thread https://github.com/notifications/unsubscribe-auth/ACDP2S5NIIFD5SMD4GQOCPLP3C6RRANCNFSM4HY63X5Q .