Zarr: why is writing to Zarr slower than copying large files to Caringo?

[agstephens]

Note that our Zarr files break each dataset into:

1. Some large files (data blocks)
2. Many tiny files (metadata and coordinates)

Some questions/thoughts:

• Can we improve write speeds by reducing the number of parallel writes?
• Can we find a way of splitting a Zarr file into:
  • small files on disk
  • large files on object store