Plan for packaging multiple MDS and FS implementations

[danielballan]

Alternative backends for the metadatastore interface were developed in separate packages: portable-mds and metadataclient. The cache-management code is cargo-culted into these repositories; bug fixes or API additions have to be applied to all three. Now that we are pretty sure these implementations are a Good Idea, I plan to move them all into this repo so they can share common code. This also reduces the packaging burden. We can separate dependencies using pip's 'extras_require' feature, e.g.:

pip install metadatastore  # includes just the core deps (e.g., doct)
pip install metadatastore[mongo]  # includes pymongo dependency
pip install metadatastore[json]  # includes mongoquery dependency
pip install metadatastore[sqlite]  # includes mongoquery dependency again (sqlite is built-in)
pip install metadatastore[client]  # includes requests dependency
pip install metadatastore[all]  # all of the above

The extras_require feature is not very widely used, but for example IPython is using it.

I would do the same for filestore and portable-fs. Speak up if you can think of a reason this is a bad idea!

cc ghost of @klauer -- look, fewer packages!

[ghost]

[CJ-Wright]

I am very much on board with this!

[klauer]

👍 👍 👍 💯

[danielballan]

https://github.com/NSLS-II/databroker/pull/145