Conflict resolution strategy support

unreadablewxy / fs-curator

Automation for the serious data hoarder that wants to have their data and use it

98 stars 2 forks source link

Conflict resolution strategy support #4

Closed unreadablewxy closed 4 years ago

unreadablewxy commented 4 years ago

Currently if an conflict is found after transforms, import is aborted.
There exists no instrument to indicate what to do when this is the case and the user is just stuck. Should provide a way to signal what to do with a conflict

Solution?

Accept a ".merge" suffix on files indicating what to do
Accept a "merge group" strategy, that combines all groups overlapping with the current group into one big group
Accept a "use existing" strategy, that accepts the file as a repeat and link it to the newly created group

unreadablewxy commented 4 years ago

Probably should let stores dictate whether they're ok with collection scoped duplicates. Probably not worth implementing target directory uniqueness?

unreadablewxy commented 4 years ago

Probably not a bad idea to create WIP files, like filesystem journals but for saving context & eliciting next steps.

Side benefit is that this approach lets us keep the core service stateless. Though we will need to support temporary observers on individual files..

Could always create the WIP files in the mono-collection and symblink it to ingestion points.. does directory/file watchers work on files opened through symblinks? strong probability of yes since the links has their own nodes.