Open prjemian opened 1 year ago
RE(bp.count([lambda2m]))
is the very basic test. Inside a plan:
def my_plan():
yield from bp.count([lambda2m])
then RE(my_plan())
Template with NeXus writing:. Since we are using the NXWriter from apstools, there is no existing documentation how to customize yet. Here is the guide demonstrating the NXWriter:
https://bcda-aps.github.io/apstools/dev/examples/fw_nxwriter.html
Instead of NXWriter, we're using NXWriterAPS since it understands the APS. The other is for use outside of APS beamlines.
In a related issue, writing the run metadata could use standardized methods. We identified in our conf call today that the standardized methods could be more general. The related issue (https://github.com/BCDA-APS/bluesky_training/issues/244) attempts to generalize the standardized methods.
@qzhang234 asked:
... how to use this with the Run Engine? Should I create a plan just like the one shown above, but replace motor_start_preprocessor with bp.count([lambda2m])), and then the plan would execute the count command and create the Nexus file?
Once this generalized method is implemented in the XPCS instrument
package, it will not be necessary to change how to the plan is written. You can use the standard plans (bp.count
, for example) and any labeled ophyd objects labeled as ad_metadata
will be written to the new stream label_start_ad_metadata
.
@prjemian I see. So should I add this part to my Bluesky plan and then proceed to the normal bp.count
?
That's not the way it works. This part:
RE.subscribe(nxwriter.receiver)
configures the session to record a new NeXus file with every run. So if you called
RE(bp.count([some_area_detector]))
you would get a NeXus file. Then if you called
RE(bp.scan([det1, det2], motor, 10, 20, 11))
you'd get another NeXus file. Same for
RE(AD_Acquire())
you'd get yet another NeXus file. No extra setup code needed.
As we discussed on Friday, the default behavior of the NXWriter() will try to copy the area detector image from the EPICS IOC-created file into the bluesky-created NeXus file. That's not what we want for XPCS. We'll need to provide some local changes to the standard NXWriter code.
Note for me: write_stream_external()
is called when the area detector HDF plugin is used. Probably this code is the one to revise or replace.
Not sure we want this as the final code in the MyNXWriter class, but it is a starting point:
def write_stream_external(self, parent, d, subgroup, stream_name, k, v):
resource_id = self.get_unique_resource(d)
fname = self.getResourceFile(resource_id)
ds = subgroup.create_dataset("file", data=fname.name)
h5addr = "/entry/data/data"
ds.attrs["target"] = ds.name
ds.attrs["source_file"] = str(fname)
ds.attrs["source_address"] = h5addr
ds.attrs["resource_id"] = resource_id
ds.attrs["shape"] = v.get("shape", "")
subgroup["external"] = h5py.ExternalLink(str(fname), h5addr)
def get_unique_resource(self, d):
# count number of unique resources (expect only 1)
resource_id_list = []
for datum_id in d:
resource_id = self.externals[datum_id]["resource"]
if resource_id not in resource_id_list:
resource_id_list.append(resource_id)
if len(resource_id_list) != 1:
# fmt: off
raise ValueError(
f"{len(resource_id_list)}"
f" unique resource UIDs: {resource_id_list}"
)
# fmt: on
return resource_id_list[0]
On my local workstation, it results in this HDF5 structure:
adsimdet --> /entry/instrument/bluesky/streams/primary/adsimdet_image
adsimdet_image:NXdata
@NX_class = "NXdata"
@signal_type = "detector"
@target = "/entry/instrument/bluesky/streams/primary/adsimdet_image"
EPOCH:NX_FLOAT64 = 1693267642.3699787
@long_name = "epoch time (s)"
@target = "/entry/instrument/bluesky/streams/primary/adsimdet_image/EPOCH"
@units = "s"
external: missing external file
@file = "/tmp/docker_ioc/iocad/tmp/adsimdet/2023/08/28/e355b37c-0d71-4dc5-a5b4_000000.h5"
@path = "/entry/data/data"
file:NX_CHAR = b'e355b37c-0d71-4dc5-a5b4_000000.h5'
@resource_id = "5357ad07-3d01-46eb-b9de-969c9ae708cf"
@shape = [ 100 1024 1024]
@source_address = "/entry/data/data"
@source_file = "/tmp/docker_ioc/iocad/tmp/adsimdet/2023/08/28/e355b37c-0d71-4dc5-a5b4_000000.h5"
@target = "/entry/instrument/bluesky/streams/primary/adsimdet_image/file"
time:NX_FLOAT64 = 0.0
@long_name = "time since first data (s)"
@start_time = 1693267642.3699787
@start_time_iso = "2023-08-28T19:07:22.369979"
@target = "/entry/instrument/bluesky/streams/primary/adsimdet_image/time"
@units = "s"
Note external: missing external file
is a local situation on my workstation. If the external file was accessible, then the external
would actually show the data in the external file.
@prjemian This is fantastic! Should I do a git pull
on Kouga?
Also, per our discussion today, how do I unsubscribe nxwriter? Not super important at this moment, just curious
Nothing to pull well use the old copy and paste technique.
To unsubscribe, we need the integer key that was returned to us when we first subscribed. Since we did not store that key, it's not easy to get it later. That's why I had you comment out that part in the setup.
revision to one of the above methods:
def write_stream_external(self, parent, d, subgroup, stream_name, k, v):
resource_id = self.get_unique_resource(d)
fname = self.getResourceFile(resource_id)
h5addr = "/entry/data/data"
ds = h5py.ExternalLink(str(fname), h5addr) # TODO: check the path
ds.attrs["target"] = ds.name
ds.attrs["source_file"] = str(fname)
ds.attrs["source_address"] = h5addr
ds.attrs["resource_id"] = resource_id
ds.attrs["shape"] = v.get("shape", "")
subgroup["value"] = ds
@qzhang234 asks (on Teams):