Incremental Store LSI

[DanEngelbrecht]

• ADDED New store index strategy allowing incremental download/upload of store index files for remote stores. Options to control and enable new strategy are --max-store-index-size and --store-index-cache-path.
• ADDED New options for cp command: --max-store-index-size
• ADDED New options for create-version-store-index command: --store-index-cache-path
• ADDED New options for downsync command: --store-index-cache-path
• ADDED New options for get command: --store-index-cache-path
• ADDED New options for print-version-usage command: --store-index-cache-path
• ADDED New options for prune-store command: --store-index-cache-path
• ADDED New options for put command: --store-index-cache-path and --max-store-index-size
• ADDED New options for upsync command: --store-index-cache-path and --max-store-index-size
• ADDED New options for validate-version command: --store-index-cache-path
• FIXED Test case usage of temporary folders
• FIXED Fixed help for --storage-uri option
• FIXED Fixed help for --s3-endpoint-resolver-uri option

[DanEngelbrecht]

Is cleaning up the local lsi cache not working properly?

[DanEngelbrecht]

TODO

• Split up index created at "init" if it goes beyond the size limit.
• Add size limit argument to "init" command
• More places we should split indexes?