search

spacetelescope / catkit2

Package for controlling testbed hardware.
https://spacetelescope.github.io/catkit2/
BSD 3-Clause "New" or "Revised" License
5 stars 0 forks source link

Option to change DataStream parameters during `.submit_data()`? #258

Open ehpor opened 3 days ago

ehpor commented 3 days ago

For many services, we want to just submit data, and change DataStream shape and dtype to whatever we are submitting. It would be nice if there is an option in DataStream.submit_data() for that. Something like:

self.images.submit_data(image, match_datastream_parameters=True)

Opinions? @raphaelpclt @lanemeier7

raphaelpclt commented 3 days ago

Hmmm I am a bit surprised. That means all the process / experiments / services reading from this datastream must be ready to adapt to a type change? Do you have a case in mind where you'd want to change type on the fly? For the shape, it's already possible to submit a smaller frame right?

lanemeier7 commented 3 days ago

Yes, modification of the original data stream size was needed for the accufiz service. It looks like historically the data was transformed and clipped a bit based radius mask resulting a change in the initial data stream which was set by sensor size in the .yml. Or you could imagine a camera with a windowing capability that can be changed on the fly

ehpor commented 3 days ago

@raphaelpclt I specifically mean, have an argument in .submit_data() that does the following:

https://github.com/spacetelescope/catkit2/blob/f90c03f5e9f03832f1a26c93483b091b4c1ccf15/catkit2/services/zwo_camera/zwo_camera.py#L189-L193

and also for the dtype.

We've been repeating this piece of code a few times now, and while I originally thought being strict was the way to go, having an opt-in which does this for you, could be useful.

ehpor commented 3 days ago

@lanemeier7 Indeed we do this for windowing cameras, dOTF intermediate results, etc... Basically anything where the shape is not known in advance and can change on the fly. You still pay a runtime cost when switching dimensions, but unless you're doing that every single frame (which you should not be doing), then it's fine.

ehpor commented 3 days ago

Or maybe make DataStream::UpdateParameters() bail out quickly when no parameter changes are needed?

Or maybe have a DataStream.match_parameters(data) function that matches the data stream to the shape and dtype of the data? But honestly, I'd prefer a flag in DataStream.submit_data() if we're going that route.

lanemeier7 commented 3 days ago

The flag sounds the cleanest to me, then it won't add any lines of code to the service logic.