search

Mirobit / Labelit

:book: Anonymize and annotate text
MIT License
7 stars 2 forks source link

Only import texts that have not been imported already #10

Closed mrweiler closed 4 years ago

mrweiler commented 4 years ago

When using the text import feature every file within the specified directory is imported. Even if the files have already been imported in a previous step. Please include a mechanism to prevent duplicates.

Mirobit commented 4 years ago

19665e63dd8c3cd0fe4cd7a98be792e1d0745d36 prevents the import function from importing duplicates and notifies the user. The import can still be successful if new texts are found.

This doesn't prevent the user from importing duplicates that exists within a csv/json file. Duplicates in the raw data are still imported. Not sure if this should be prevented.

mrweiler commented 4 years ago

This is exactly what I wanted:

19665e6 prevents the import function from importing duplicates and notifies the user. The import can still be successful if new texts are found.

Not sure, if this would be important/useful at some point, but I consider it a different issue:

This doesn't prevent the user from importing duplicates that exists within a csv/json file. Duplicates in the raw data are still imported. Not sure if this should be prevented.

The import function does not recognize any edits/updates to the original text files. Not sure if this would be important/useful. Also not sure if it belongs to this or a completely new issue.

Mirobit commented 4 years ago

The import function does not recognize any edits/updates to the original text files. Not sure if this would be important/useful. Also not sure if it belongs to this or a completely new issue.

I haven't thought about this. It could be a useful feature. I will add this to the backlog.