G-Node / gin-cli

Command line client for GIN
https://gin.g-node.org
Other
12 stars 6 forks source link

File metadata write is slow #269

Open achilleas-k opened 5 years ago

achilleas-k commented 5 years ago

The metadata filename feature was added in 604176d4642599d496deab6bbc1fbf9b8528524b (Apr 2018).

Feature description: After annex add is finished, the client writes the filename to the content metadata so that the annex content key is always associated with the name of the file it was added as. This is useful for knowing which file is being transferred when uploading old versions of files (when multiple commits happen between uploads).

Problem: The metadata writing can take a while when a large number of files is being added, so printing this information is useful.

Solutions:

Additional info: The current solution doesn't take into account that keys can be mapped to multiple file names. Filenames that have the same content will share the same content key (same hash) and currently this isn't handled in any way. Instead, the last filename to be added simply overwrites any existing value. If we support multiple filenames, we would have to have a way of choosing which one to display during a git annex copy.