search

NSLS-II / metadatastore

DEPRECATED: Incorporated into https://github.com/NSLS-II/databroker
Other
2 stars 11 forks source link

API: Apply same sorting to v0 as was applied to v1 previously. #213

Closed danielballan closed 8 years ago

danielballan commented 8 years ago

We agreed on a sorting for the find_* methods in #204, but #204 did not apply that sorting to core_v0.py.

This obviously makes validation easier, as the results from v0 and v1 can be zipped and compared.

danielballan commented 8 years ago

@arkilic While we're on the subject, have these sorting choices propagated into metadataservice yet?

ghost commented 8 years ago

@danielballan I had earlier on but I need to walk through since quite a few things have changed since December

danielballan commented 8 years ago

If very much has changed, one possibility it make metadataserver a web-wrangling frontend for metadatastore with no new db logic.

licode commented 8 years ago

Can you briefly mention the difference between v0 and v1? I didn't follow up on this these days, but kind of want to pick it up.

danielballan commented 8 years ago

Sure. v0 has two unique IDs: a 'uid' string created by us and referenced by our codebases and an '_id' ObjectId created by mongo and used by it internally. In v1, there is only the 'uid'. This gave a performance boost.

danielballan commented 8 years ago

@licode Also see https://github.com/NSLS-II/event-model and the illustrated documentation linked from there.

licode commented 8 years ago

Thanks, and good to know. The design on v1 is definitely optimal.

ghost commented 8 years ago

Right, mdstore as it is did NOT exist when I was writing the service so I did not get to use it in the beginning. As a matter of fact, it started using uids as foreign keys in October. The indexing work I see here is on the cursors, which is changing directions of indices in the server side and a very simple fix. Using metadatastore library(mds) on the server side has complexities. For instance, every exception mds raises has to be caught and translated into its corresponding HTTPError. We also need to make sure if things go wrong in mds, the server will not crash or keep hanging/cause performance issues, which requires rigorous testing. Having said that, it has obvious advantages that I do not really need to list here. I say this is a good idea and task for a refactor that we were all aware that was coming but I am not sure if I can start tackling it prior to mid-May.

tacaswell commented 8 years ago

The v1 of metadatastore primarily exists as a testing/verification/debugging tool for the service/client layer.

licode commented 8 years ago

I see. In the mds.py, we can easily switch between those two versions, smart design.