APS-USAXS / usaxs-bluesky-ended-2023

Bluesky instrument for USAXS
0 stars 0 forks source link

BlackFly2 does not work. #573

Open jilavsky opened 1 year ago

jilavsky commented 1 year ago

Looks like we have misconfigured BlackFly 2 used for collecting images of samples before exposures. I get this:

W Tue-13:25:09 - Could not take sample image:path=/share1/USAXS_data/2022-10/11_01_Stability21keV/11_01_Stability21keV_saxs, file=Tape_BLANK, order#=894, exc=The status Status(obj=EpicsSignalWithRBV(read_pv='9idFLY2:JPEG1:Capture_RBV', name='blackfly_optical_jpeg1_capture', parent='blackfly_optical_jpeg1', value=0, timestamp=1667327049.6286075, auto_monitor=True, string=False, write_pv='9idFLY2:JPEG1:Capture', limits=False, put_complete=False), done=False, success=False) has failed. To obtain more specific, helpful errors in the future, update the Device to use set_exception(...) instead of _finished(success=False).
prjemian commented 1 year ago

Could not take sample image

is not very specific. Any clue what is the real problem?

prjemian commented 1 year ago

Ah, Status(obj=EpicsSignalWithRBV(read_pv='9idFLY2:JPEG1:Capture_RBV', looks like the file writing plugin (JPEG) is not primed. Correct?

jilavsky commented 1 year ago

Yes, was not primed. Camera was being used for focusing and sampel survey, but without jpg writer enabled and so that did not have anything in it. I manually saved one jpg image. Yes, it is working after I manually saved image. I am getting the jpg images now. This need for manual priming is annoying... I may remember eventually, but it is annoying. Can we develop some priming procedure run automatically, if needed?

prjemian commented 1 year ago

There is an example of checking if a camera plugin is not primed in the training repository:

if iconfig.get("ALLOW_AREA_DETECTOR_WARMUP", False):
    if not AD_plugin_primed(adsimdet.hdf1):
        AD_prime_plugin2(adsimdet.hdf1)

USAXS might do similarly. The AD_plugin_primed() check allows bluesky to start most of the time without priming.

prjemian commented 1 year ago

note:

from apstools.devices import AD_plugin_primed
from apstools.devices import AD_prime_plugin2
prjemian commented 1 year ago

This should work for any of these AD file writing plugins, HDF5, JPEG, TIFF.

jilavsky commented 1 year ago

Not sure how this is related, but recently I had issues with BS hanging in the image collection here - and after control-C and RE.abort() BS continued in data collection. Restarted softioc and saved manually image to make sure jpg writer is primed with image. After this I start getting:

W Fri-10:44:37 - Could not take sample image:path=/share1/USAXS_data/2022-11/11_18_Gadikota/11_18_Gadikota_saxs, file=WaterBlank_12min, order#=276, exc=Device MyPointGreyDetectorJPEG(prefix='9idFLY2:', name='blackfly_optical', read_attrs=['jpeg1'], configuration_attrs=['cam', 'cam.acquire_period', 'cam.acquire_time', 'cam.image_mode', 'cam.manufacturer', 'cam.model', 'cam.num_exposures', 'cam.num_images', 'cam.trigger_mode', 'jpeg1']) is already staged. Unstage it first.

So now I need to unstage the device first. I suspect this was due to BS hanging in staging process and now not being able to unstage itself and restage itself. This staging seems very fragile, what is the problem of staging already staged device? Makes no sense to this stupid scientist...

prjemian commented 1 year ago

staging already staged device

Calling device.stage(); device.stage() will raise an exception.

jilavsky commented 1 year ago

Hmmm, simple solution then.

def device.stage() 
      if device.isStaged() is True : 
            device.unStage()
      ..... 

To be fair, the device stayed staged likely after I had to kill the thread which is collecting images and this left things in wrong state. Obviously, RE.abort() did not unstage the device (Q: should? Is this realistic action to ask from BS?). Correct solution is to restart softioc and BS to clear all out, but as we know, that may not be the thing user can do at 2am on Sunday morning... Not sure how much effort we need to put in this relatively rare case. But it is enough of trouble, that for current users I disabled jpg image taking to make sure this does not stop their data collection while they are running over weekend now.