Large table data

[constantinpape]

For HCS plates with segmentations (or similar data), we will have a large amount of table data. At some point it becomes impractical to store them on github. And it will also become inefficient to store them in a text file format like csv / tsv. We should think about supporting more table formats:

• tables stored in zarr/n5
• tables stored in a database (like we have it in the plateViewer already)
  • in plateViewer we use mongoDB
  • when we discussed this with @titansmc before, he suggested looking into https://couchdb.apache.org/

[constantinpape]

Databases on s3: https://towardsdatascience.com/turn-amazon-s3-into-a-spatio-temporal-database-40f1a210e943

[jkh1]

For tabular data, I'd suggest using SQLite. No need for a server or complicated setup.