bluesky / event-model

data model for event-based data collection and analysis
https://blueskyproject.io/event-model
BSD 3-Clause "New" or "Revised" License
13 stars 30 forks source link

ENH: scrub and offset data #211

Open tacaswell opened 3 years ago

tacaswell commented 3 years ago

This is primarily useful if you want to obscure the origin of some data.

tacaswell commented 3 years ago

Example usage.


from databroker._drivers.msgpack import BlueskyMsgpackCatalog
from suitcase.msgpack import Serializer

def sanitize_start(doc):
    doc["beamline_id"] = "MAD"
    for k in ("group", "project"):
        doc.pop(k, None)

    doc["sample"]["type"] = "Cool Material"
    return doc

cat = BlueskyMsgpackCatalog('/tmp/some_source/*.msgpack')

target = Serializer('/tmp/targetl')

for name, doc in scrub([cat[i] for i in cat], 0, 1000, sanitize_start):
    target(name, doc)

test = BlueskyMsgpackCatalog("/tmp/*msgpack")