See post in #112 and copied below from Slack for details.
How do you absolutely guarantee that a new feature will come up? You do a release that doesn't include that feature.
...during the last run, I stared a bit at monitor data and had a minor brainwave. For a long set of reasons that I outline in #112, metadata loading for SST1 RSoXS's sparse data is fairly unreliable for short exposures, and even for some long ones for the beamstop in particular (not that that's important, right? ...right?). On a Friday afternoon, I decided to try to whip together a little fix for this problem.
In brief: we previously were using an overestimate of when the shutter was open, which frequently included dark points in some (many?) individual energy points for counters that are downstream of the shutter (beamstop and TEY). In my new version, for these counters, we just treat the counter signal itself as a square wave and trim points from it directly. An example result is below, old code in blue producing ~unusable data and new code in orange, producing what I hope you'll find to be acceptable data. This is from a 1s exposure WAXS scan taken a couple days ago.
To test:
git checkout 112-userwarning-error-while-time-integrating-monitors
your PyHyperScattering.__version__ should be 0.2.3+3.g4135a3e
import PyHyperScattering
load = PyHyperScattering.load.SST1RSoXSDB()
run = loadRun(#####,return_dataset=True)
run will now be an xarray.Dataset that contains all the monitors, interpolated onto the energy (or whatever) points of the scan, screenshot attached.
You can plot the monitors using Dataset syntax like so:
(run['RSoXS Au Mesh Current']/run['WAXS Beamstop']).unstack('system').plot(x='energy',hue='polarization')
NEXAFS plot attached.
To get the detector images out for further processing, simply use run['Wide Angle CCD Detector_image'] instead of run that you would use if the dataset flag was off.
If you run into issues loading monitors, you can troubleshoot with the load.loadMonitors function. The new kwarg directLoadPulsedMonitors=True causes the new direct loading mechanism to overwrite the old method with the new. If you set it to False you'll get a dataset with pl_WAXS Beamstop (say) for the new version and just WAXS Beamstop for the old. For example, the way I made the first attachment is:
You should also feel free to message with the scan number.
I suspect this will fail, maybe badly, for very short exposures (<0.5 s). It might be possible that better signal processing lets you recover these. But this gets usable NEXAFS out of a lot of scans now.
See post in #112 and copied below from Slack for details.
How do you absolutely guarantee that a new feature will come up? You do a release that doesn't include that feature.
...during the last run, I stared a bit at monitor data and had a minor brainwave. For a long set of reasons that I outline in #112, metadata loading for SST1 RSoXS's sparse data is fairly unreliable for short exposures, and even for some long ones for the beamstop in particular (not that that's important, right? ...right?). On a Friday afternoon, I decided to try to whip together a little fix for this problem.
In brief: we previously were using an overestimate of when the shutter was open, which frequently included dark points in some (many?) individual energy points for counters that are downstream of the shutter (beamstop and TEY). In my new version, for these counters, we just treat the counter signal itself as a square wave and trim points from it directly. An example result is below, old code in blue producing ~unusable data and new code in orange, producing what I hope you'll find to be acceptable data. This is from a 1s exposure WAXS scan taken a couple days ago.
To test:
git checkout 112-userwarning-error-while-time-integrating-monitorsyourPyHyperScattering.__version__should be0.2.3+3.g4135a3erunwill now be anxarray.Datasetthat contains all the monitors, interpolated onto the energy (or whatever) points of the scan, screenshot attached.You can plot the monitors using Dataset syntax like so:
(run['RSoXS Au Mesh Current']/run['WAXS Beamstop']).unstack('system').plot(x='energy',hue='polarization')NEXAFS plot attached.To get the detector images out for further processing, simply use
run['Wide Angle CCD Detector_image']instead ofrunthat you would use if the dataset flag was off.If you run into issues loading monitors, you can troubleshoot with the
load.loadMonitorsfunction. The new kwargdirectLoadPulsedMonitors=Truecauses the new direct loading mechanism to overwrite the old method with the new. If you set it to False you'll get a dataset with pl_WAXS Beamstop (say) for the new version and just WAXS Beamstop for the old. For example, the way I made the first attachment is:You should also feel free to message with the scan number.
I suspect this will fail, maybe badly, for very short exposures (<0.5 s). It might be possible that better signal processing lets you recover these. But this gets usable NEXAFS out of a lot of scans now.