Support partial zarr upload

[yarikoptic]

Original use-case

• https://github.com/dandi/dandi-cli/issues/1474

to modify some metadata files within zarr. Zarr itself might be huge (GBs) so it is a big waste to require download of the entire zarr folder to just update some metadata files. So relates also to ability for partial download

• dandi/dandi-archive#1461

dandi upload if operating on a .zarr folder, should gain an option so upload only considers paths present but does not assume that it is "all of zarr" and remove the rest from remote zarr. Then it needs to finalize zarr update with full manifest as with some files/folders of zarr present only in archive, and not locally.

I think it should be --zarr-mode full,partial-full,partial-lean. Modes

• full -- current behavior (pretty much a sync IIRC)
• partial-full -- if encountering zarr sub-folder locally, it would pretty much do full on that subfolder -- i.e. it would remove remote files under that subfolder if not present locally. As a "side-effect" -- if encounters an empty sub-folder locally -- it would remove all the zarr subfiles potentially present with that folder prefix on archive (thus it would be easy to prune some undesired remote folders).
• partial-lean -- it would only concern with updates of files it encounters.

[kabilar]

Thanks @yarikoptic. Hi @yarikoptic @jwodder, following up from our discussion yesterday, could you please prioritize this issue? Thank you.