Closed coretl closed 4 months ago
@coretl what is the classname and location? should it be in core
, epics
, or somewhere else? should I make a simprovider
like over there? https://github.com/dls-controls/bluefly/blob/master/bluefly/simprovider.py or just use simsignal backend?
Please can you make:
ophyd_async.sim.PatternGenerator
a class that is not a Device
, but does the logic from https://github.com/dls-controls/bluefly/blob/master/bluefly/areadetector.py with methods like open_file()
, set_exposure()
, set_x()
, set_y()
, write_image_to_file()
, etc.ophyd_async.sim.SimPatternDetectorControl
- subclass of ophyd_async.core.DetectorControl
that takes a PatternGenerator
at __init__
and sets the exposure and starts a task writing images to fileophyd_async.sim.SimPatternDetectorWriter
- subclass of ophyd_async.core.DetectorWriter
that takes a PatternGenerator
at __init__
and opens and closes file and reports how many frames are written on requestophyd_async.sim.SimPatternDetector
- subclass of ophyd_async.core.StandardDetector
that makes the above 3 itemsAt some point in the future we will make a SimSample
that has x and y motors which call set_x
and set_y
on the PatternGenerator
The bluefly
code is for the maths only, the rest of the framework has been superceded by ophyd-async
Must be supplied instances of classes that inherit from DetectorControl and DetectorData, to dictate how the detector will be controlled (i.e. arming and disarming) as well as how the detector data will be written (i.e. opening and closing the writer, and handling data writing indices).
that's the description of ophyd_async.core.StandardDetector
which I take to mean that the Control and Writer objects need to be passed to the SimPatternDetector
constructor, not made inside of it
The aim is to make something that looks like: https://github.com/DiamondLightSource/dodal/blob/directory_provider/src/dodal/devices/areadetector/pilatus.py
so subclass, then make the Control and Writer in __init__
passing them to the superclass __init__
I'm slowly getting more familiar with asyncio
, h5py
and a couple of things are not clear to me.
state
spread out? One way is for PatternGenerator to holder exposure, x, y, image array and the file handle, writer holding indices_written. or should PatternGenerator have no state?1. which processes are initialized as Tasks and which not?
I think I gave you the wrong link, it shouldn't be areadetector.py
from bluefly, but https://github.com/dls-controls/bluefly/blob/master/bluefly/areadetector_sim.py. There aren't any Tasks in there.
2. should I use DirectoryProvider?
Yes please, to know where to write the file.
3. what should the deadtime value be?
It doesn't really matter, but let's make it 1ms so it looks like a real detector.
4. how is the holding of `state` spread out? One way is for PatternGenerator to holder exposure, x, y, image array and the file handle, writer holding indices_written. or should PatternGenerator have no state?
PatternGenerator should hold the state.
5. why should patternGenerator have all those methods: open & write to file and set x, y?
So that:
in the comment before you mention:
ophyd_async.sim.SimPatternDetectorControl - subclass of ophyd_async.core.DetectorControl that takes a PatternGenerator at init and sets the exposure and starts a task writing images to file
so I assumed that there's a Task there somewhere
RuntimeError: Unable to start swmr writing (file superblock version - should be at least 3)
working on solving this
not sure what is the reasonf ror the logic in this method. in what occasion would self._file
be empty (uninitalized) here? why not initalize in init / open?
because you can't get the full filename the HDF writer will write until the first frame comes in...
and how does that impact that stream resources are yielded and only later stream data?
this is quite confusing
should then the self.directory_provider
be initialized in init?
I was assuming that open_file
is always called first
and how does that impact that stream resources are yielded and only later stream data?
We defer yielding StreamResource until the file is open and then emit it along with the first StreamDatum
should then the
self.directory_provider
be initialized in init? I was assuming thatopen_file
is always called first
Do you mean should we call self._directory_provider()
in __init__
? No, we need to call it once per file we need to open
To clarify, this problem is related to handling of NumPy's Unicode string type. HDF5 (and h5py) don't support this type. Details here: h5py: What about NumPy’s U type?
potential numpy types issue
Failed validating 'additionalProperties' in schema['patternProperties']['^([^./]+)$']: {'additionalProperties': False, 'patternProperties': {'^([^./]+)$': {'$ref': '#/definitions/DataType'}}, 'title': 'DataType'}
typecheck for descriptor fails
On instance['data_keys']: {'/entry/data/data': {'dtype': 'array', 'external': 'STREAM:', 'object_name': 'PATTERN1', 'shape': (1, 240, 320), 'source': 'sim:///entry/data/data'}, '/entry/sum': {'dtype': 'array', 'external': 'STREAM:', 'object_name': 'PATTERN1', 'shape': (1,), 'source': 'sim:///entry/sum'}}
File "/scratch/xma12127/projects/forks/ophyd-async/src/ophyd_async/sim/SimPatternDetectorControl.py", line 55, in _coroutine_for_image_writing await self.pattern_generator.write_image_to_file() File "/scratch/xma12127/projects/forks/ophyd-async/src/ophyd_async/sim/PatternGenerator.py", line 186, in write_image_to_file self._handle_for_h5_file[DATA_PATH].resize(target_dimensions) File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper File "/venv/lib/python3.9/site-packages/h5py/_hl/group.py", line 357, in __getitem__ oid = h5o.open(self.id, self._e(name), lapl=self._lapl) File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper File "h5py/h5o.pyx", line 189, in h5py.h5o.open KeyError: 'Unable to synchronously open object (component not found)'
threads here do not succeed. did not expect the .resize
line to fail of all the possible ones
CI fails tests with: =========================== short test summary info ============================ FAILED tests/core/test_flyer.py::test_hardware_triggered_flyable - RuntimeError: Collect now emits EventPages (stream=False), so emitting Events (stream=True) is no longer supported ================== 1 failed, 185 passed, 1 skipped in 11.77s =================== ERROR: InvocationError for command /opt/hostedtoolcache/Python/3.10.13/x64/bin/pytest --cov=ophyd_async --cov-report term --cov-report xml:cov.xml (exited with code 1) ___________________________________ summary ____________________________________ ERROR: pytest: commands failed
and lint with nested conftest
Based on the code here: https://github.com/dls-controls/bluefly/blob/master/bluefly/areadetector_sim.py