search

European-XFEL / karabo-bridge-py

Tools to allow data exchange with Karabo, in particular streaming of data
BSD 3-Clause "New" or "Revised" License
9 stars 4 forks source link

Option to transmit recorded data #63

Closed FilipeMaia closed 3 years ago

FilipeMaia commented 3 years ago

Feature request to add an option to read a previously recorded data file (.h5) and send if out through the bridge.

tmichela commented 3 years ago

Hi @FilipeMaia

The feature already exists as part of the EXtra-data package. See the streaming documentation for usage details.

If your files are EuXFEL formated, you can simply use the CLI tool karabo-bridge-serve-files. If that's not the case, you can use extra_data.export.ZMQStreamer to send data in the karabo bridge format.

You can try it on Maxwell using our provided environment with module load exfel exfel_anaconda3 or install extra-data with pip install extra-data in your own.

Let me know if that helps,

Thomas

FilipeMaia commented 3 years ago

Hi @tmichela

I was not aware. That's very useful! Maybe you should point this out in the help or something.

Cheers, Filipe

takluyver commented 3 years ago

Just to add, we also have an internal tool called karabo-bridge-recorder, which can capture the ZMQ messages and play them back in exactly the same format, for testing code with. We don't want anyone to use that to save scientific data, but if you want to capture some samples of bridge data purely for testing purposes, ping us on da-support@xfel.eu and we can help you work it out.

tmichela commented 3 years ago

Thanks for the feedback @FilipeMaia

Maybe you should point this out in the help or something.

Currently the documentation about the karabo bridge protocol is part of our general EuXFEL data analysis documentation as it covers different packages.

And I agree it is not particularly user friendly to find information in that case. I planed to move the documentation here a while back (#59), I'll resume working on that part.