search

NSLS-II-CSX / profile_collection

BSD 3-Clause "New" or "Revised" License
0 stars 5 forks source link

FIX: Generate a new Resource document on resume. #8

Closed danielballan closed 5 years ago

danielballan commented 5 years ago

Submitted for testing.

licode commented 5 years ago

what's the status of this PR? already tested?

danielballan commented 5 years ago

I am not sure. I have followed up on the email thread where I first sent the first to @cmazzoli.

stuartcampbell commented 5 years ago

Just quoting the email from @danielballan here for transparency and future search-ability!

Testing should be straightforward. First check that you can reproduce the issue. Start bsui, run a lengthy count, wait for acquisition to begin, close a shutter to initiate suspension, and reopen the shutter to lift the suspension. Attempt to access the data: at this point you should run into an error.

Having reproduced the issue, now check if the proposed changes fix it. Check out the PR branch like so:

cd <YOUR_PROFILE_DIRECTORY>
git fetch origin
git checkout new-resource-on-resume

Run through the same procedure again, and confirm that this time you can access if that data. If that works, please merge the pull request. Let me know if you have any questions about the procedure or if the results do not come out as we expect.

danielballan commented 5 years ago

The test above was not done, and the problem was hit again with users present. We then performed the test, and @tacaswell and I made some necessary additional fixes in a rush. @cmazzoli or @wen-hu, would you please push those changes to this branch?

tacaswell commented 5 years ago

as part of the work on Mar 11 we rebased this branch on top of CSX's current master and then added one more commit, force-pushed to update this PR to match what is on the beam line.

gwbischof commented 5 years ago

aDiscussed this with @cmazzoli. This was tested a few weeks ago and we agreed on merging it.

mrakitin commented 5 years ago

See https://github.com/NSLS-II-CSX/xf23id1_profiles/commit/754051e2c35a903ccb88b8f5bba9fd71577293b9 for the additional work (with @tacaswell's remote assistance today) to make it fully work.

Test run:

[xf23id1@xf23id-ws3 ~/.ipython/profile_collection/startup/csx1/devices]$ bsui

Versions of DAMA software:

    - Bluesky    : v1.5.2
    - Ophyd      : v1.3.2
    - Databroker : v0.12.2

Links to Bluesky and Databroker tutorials:

    - http://nsls-ii.github.io/bluesky/tutorial.html
    - http://nsls-ii.github.io/databroker/tutorial.html

If you get lost or confused, then we want to know! We have a friendly chat channel:

    https://gitter.im/NSLS-II/DAMA

or you can file a bug to let us know where our documentation could be made more clear:

    https://github.com/NSLS-II/Bug-Reports/issues

bsui is running these commands now to start an interactive computing environment for data acquisition:
    $ conda activate collection-2019-2.0-csx
    $ ipython --profile=collection --IPCompleter.use_jedi=False

Activating auto-logging. Current session state plus future input saved.
Filename       : /epics/xf/23id/ophyd/logs/xf23id1_log.py
Mode           : append
Output logging : False
Raw input log  : False
Timestamping   : False
State          : active
Python 3.6.6 |Anaconda, Inc.| (default, Oct  9 2018, 12:34:16)
Type 'copyright', 'credits' or 'license' for more information
IPython 7.2.0 -- An enhanced Interactive Python. Type '?' for help.

IPython profile: collection
Using matplotlib backend: Qt5Agg
Loading metadata history from /home/xf23id1/.config/bluesky/bluesky_history.db

In [1]: RE.clear_suspenders()

In [2]: %run -i /home/xf23id1/Beamline/Commissioning/2019_04/startup.py

In [3]: RE(mv(fccd.exposure, (1,0.5,5)))
Out[3]: ()

In [4]: RE(ct([fccd],num=3))

Transient Scan ID: 116810     Time: 2019-04-19 12:26:28
Persistent Unique Scan ID: '74cfbec9-9e99-45d6-a93f-0325af4ac688'
New stream: 'baseline'
fcA 'deferred pause' has been requested. The RunEngine will pause at the next checkpoint. To pause immediately, hit Ctrl+C again in the next 10 seconds.      | 1.0/5.0 [00:01<00:04,  1.16s/images]
Deferred pause acknowledged. Continuing to checkpoint.
New stream: 'primary'
+-----------+------------+-------------------+-------------------+-------------------+
|   seq_num |       time | fccd_stats1_total | fccd_stats2_total | fccd_stats5_total |
+-----------+------------+-------------------+-------------------+-------------------+
|         1 | 12:26:41.6 |             -1559 |             -1769 |            146051 |
Pausing...
---------------------------------------------------------------------------
RunEngineInterrupted                      Traceback (most recent call last)
~/Beamline/Commissioning/2019_04/startup.py in <module>
----> 1 RE(ct([fccd],num=3))

/opt/conda_envs/collection-2019-2.0-csx/lib/python3.6/site-packages/bluesky/run_engine.py in __call__(self, *args, **metadata_kw)
    757
    758             if self._interrupted:
--> 759                 raise RunEngineInterrupted(self.pause_msg) from None
    760
    761         return tuple(self._run_start_uids)

RunEngineInterrupted:
Your RunEngine is entering a paused state. These are your options for changing
the state of the RunEngine:

RE.resume()    Resume the plan.
RE.abort()     Perform cleanup, then kill plan. Mark exit_stats='aborted'.
RE.stop()      Perform cleanup, then kill plan. Mark exit_status='success'.
RE.halt()      Emergency Stop: Do not perform cleanup --- just stop.

In [5]: RE.resume()
|         2 | 12:26:53.5 |             -2773 |              -277 |           -973664 |
|         3 | 12:26:59.3 |              2721 |              3170 |            618860 |
+-----------+------------+-------------------+-------------------+-------------------+
generator count ['74cfbec9'] (scan num: 116810)
Out[5]: ['74cfbec9-9e99-45d6-a93f-0325af4ac688']

In [6]: hdr = db[-1]

In [7]: [d for n, d in hdr.documents() if n =='datum']
Out[7]:
[{'resource': '9eb3752d-3588-4397-b584-fbf465fc96e6',
  'datum_id': '9eb3752d-3588-4397-b584-fbf465fc96e6/0',
  'datum_kwargs': {'point_number': 0}},
 {'resource': '23ca585b-6a72-4c0a-bd16-57744fc3583b',
  'datum_id': '23ca585b-6a72-4c0a-bd16-57744fc3583b/0',
  'datum_kwargs': {'point_number': 0}},
 {'resource': '23ca585b-6a72-4c0a-bd16-57744fc3583b',
  'datum_id': '23ca585b-6a72-4c0a-bd16-57744fc3583b/1',
  'datum_kwargs': {'point_number': 1}}]

In [8]: [d for n, d in hdr.documents() if n =='resource']
Out[8]:
[{'spec': 'AD_HDF5',
  'resource_path': 'fccd_data/2019/04/19/f9bcf1e3-4466-42de-9e0e_000000.h5',
  'root': '/GPFS/xf23id/xf23id1',
  'resource_kwargs': {'frame_per_point': 5},
  'path_semantics': 'posix',
  'uid': '9eb3752d-3588-4397-b584-fbf465fc96e6',
  'run_start': '74cfbec9-9e99-45d6-a93f-0325af4ac688',
  'id': '9eb3752d-3588-4397-b584-fbf465fc96e6'},
 {'spec': 'AD_HDF5',
  'resource_path': 'fccd_data/2019/04/19/f9bcf1e3-4466-42de-9e0e_000001.h5',
  'root': '/GPFS/xf23id/xf23id1',
  'resource_kwargs': {'frame_per_point': 5},
  'path_semantics': 'posix',
  'uid': '23ca585b-6a72-4c0a-bd16-57744fc3583b',
  'run_start': '74cfbec9-9e99-45d6-a93f-0325af4ac688',
  'id': '23ca585b-6a72-4c0a-bd16-57744fc3583b'}]

In [9]: imgs = np.array(list(hdr.data('fccd_image')))

In [10]: imgs
Out[10]:
array([[[[-27453, -27636, -27616, ..., -27524, -27579, -27418],
         [  5124,   5143,   5114, ...,   5759,   5756,   5755],
         [  5037,   5054,   5061, ...,   5665,   5656,   5696],
         ...,
         [  5834,   5868,   5837, ...,   5231,   5205,   5290],
         [  5948,   5932,   5939, ...,   5300,   5293,   6033],
         [-26977, -27413, -27384, ..., -10693,  -9568,  -8193]],
...