Mnt prep for demo - Githubissues

tacaswell commented 3 years ago

Work to get queueserver to start up against the PDF beamline profile.

So far this starts against spoof beamline, but can not run scans yet due to not having a good enough fake shutter.

~/.config/acq/pdf.yml

archive_base_dir_name: .userbeamtimearchive
archive_root_dir: /mnt/data/bnl/xpdacq_special/archive
base_dir: /mnt/data/bnl/xpdacq_special/data
beamline_host_name: [jupiter]
beamline_id: 28-ID-1
blconfig_dir_name: xpdConfig
blconfig_name: xpd_beamline_config.yml
calib_config_name: xpdAcq_calib_info.yml
dark_window: 0.1
facility: NSLS-II
frame_acquire_time: 0.1
glbl_yaml_name: glbl.yml
group: XPD
home_dir_name: user_data
image_field: pe1c_image
owner: PDF
simulation: false
shutter_conf: {close: 0, open: 1}
mask_kwargs:
  edge: 30
  lower_thresh: 0.0
  bs_width: 13
  tri_offset: 13
  v_asym: 0
  alpha: 3.0
dark_field_key : sc_dk_field_uid
det_image_field : pe1_image
exp_broker_name: pdf
outbound_proxy_address: 'xf28id1-ca1:5578'
inbound_proxy_address: 'xf28id1-ca1:5577'
shutter_sleep: 0
diffraction_dets: ['pe1c', 'pe1', 'pe2c', 'pe2','dexela_image', 'dexela']
radiograph_names : ['blackfly_det']
radiogram_dets: ['blackfly_det']
image_fields: ['pe1c_image', 'pe2c_image', 'dexela_image', 'dexela']

tacaswell commented 3 years ago

total 0
drwxr-xr-x 4 tcaswell tcaswell 40 Feb 26 16:21 .
drwxr-xr-x 3 tcaswell tcaswell 18 Feb 26 16:21 ..
drwxr-xr-x 7 tcaswell tcaswell 95 Feb 26 16:21 user_data
drwxr-xr-x 2 tcaswell tcaswell 37 Feb 26 16:27 xpdConfig

./user_data:
total 0
drwxr-xr-x 7 tcaswell tcaswell 95 Feb 26 16:21 .
drwxr-xr-x 4 tcaswell tcaswell 40 Feb 26 16:21 ..
drwxr-xr-x 2 tcaswell tcaswell  6 Feb 26 16:21 Import
drwxr-xr-x 3 tcaswell tcaswell 42 Feb 27 16:20 config_base
drwxr-xr-x 2 tcaswell tcaswell  6 Feb 26 16:21 tiff_base
drwxr-xr-x 2 tcaswell tcaswell 20 Feb 26 16:21 userAnalysis
drwxr-xr-x 2 tcaswell tcaswell  6 Feb 26 16:21 userScripts

./user_data/Import:
total 0
drwxr-xr-x 2 tcaswell tcaswell  6 Feb 26 16:21 .
drwxr-xr-x 7 tcaswell tcaswell 95 Feb 26 16:21 ..

./user_data/config_base:
total 4
drwxr-xr-x 3 tcaswell tcaswell 42 Feb 27 16:20 .
drwxr-xr-x 7 tcaswell tcaswell 95 Feb 26 16:21 ..
-rw-r--r-- 1 tcaswell tcaswell 23 Feb 27 16:23 .sample_order.yml
drwxr-xr-x 4 tcaswell tcaswell 71 Feb 26 16:26 yml

./user_data/config_base/yml:
total 12
drwxr-xr-x 4 tcaswell tcaswell   71 Feb 26 16:26 .
drwxr-xr-x 3 tcaswell tcaswell   42 Feb 27 16:20 ..
-rw-r--r-- 1 tcaswell tcaswell   95 Feb 27 17:25 bt_bt.yml
-rw-r--r-- 1 tcaswell tcaswell 4896 Feb 27 17:25 glbl.yml
drwxr-xr-x 2 tcaswell tcaswell    6 Feb 26 16:21 samples
drwxr-xr-x 2 tcaswell tcaswell    6 Feb 26 16:21 scanplans

./user_data/config_base/yml/samples:
total 0
drwxr-xr-x 2 tcaswell tcaswell  6 Feb 26 16:21 .
drwxr-xr-x 4 tcaswell tcaswell 71 Feb 26 16:26 ..

./user_data/config_base/yml/scanplans:
total 0
drwxr-xr-x 2 tcaswell tcaswell  6 Feb 26 16:21 .
drwxr-xr-x 4 tcaswell tcaswell 71 Feb 26 16:26 ..

./user_data/tiff_base:
total 0
drwxr-xr-x 2 tcaswell tcaswell  6 Feb 26 16:21 .
drwxr-xr-x 7 tcaswell tcaswell 95 Feb 26 16:21 ..

./user_data/userAnalysis:
total 4
drwxr-xr-x 2 tcaswell tcaswell   20 Feb 26 16:21 .
drwxr-xr-x 7 tcaswell tcaswell   95 Feb 26 16:21 ..
-rw-r--r-- 1 tcaswell tcaswell 1426 Feb 26 16:21 Ni24.D

./user_data/userScripts:
total 0
drwxr-xr-x 2 tcaswell tcaswell  6 Feb 26 16:21 .
drwxr-xr-x 7 tcaswell tcaswell 95 Feb 26 16:21 ..

./xpdConfig:
total 4
drwxr-xr-x 2 tcaswell tcaswell 37 Feb 26 16:27 .
drwxr-xr-x 4 tcaswell tcaswell 40 Feb 26 16:21 ..
-rw-r--r-- 1 tcaswell tcaswell 58 Feb 26 16:47 xpd_beamline_config.yml

(sys39) /m/d/b/x/data 
✔ 12:31:01 $ cat user_data/config_base/.sample_order.yml
0: bob.yml
1: bill.yml

(sys39) /m/d/b/x/data 
✔ 12:31:40 $ cat user_data/config_base/yml/bt_bt.yml
bt_experimenters: []
bt_piLast: Caswell
bt_safN: '123456'
bt_uid: 76f3b555
bt_wavelength: null

(sys39) /m/d/b/x/data 
✔ 12:32:19 $ cat user_data/config_base/yml/glbl.yml
_dark_dict_list:
- acq_time: 5
  exposure: 5
  timestamp: 1614460063.2800422
  uid: 37d192fb-df2c-44a9-b41b-2a05c9751247
- acq_time: 5
  exposure: 5
  timestamp: 1614460242.3911414
  uid: cf374ae7-2ed5-4e5d-9563-f4841a3759e9
- acq_time: 5
  exposure: 5
  timestamp: 1614460464.3035522
  uid: 5fc90d2e-ef96-4ca7-9207-ae72121923d0
- acq_time: 5
  exposure: !!python/object/apply:numpy.core.multiarray.scalar
  - &id001 !!python/object/apply:numpy.dtype
    args:
    - f8
    - false
    - true
    state: !!python/tuple
    - 3
    - <
    - null
    - null
    - null
    - -1
    - -1
    - 0
  - !!binary |
    AAAAAAAAFEA=
  timestamp: 1614460566.1485345
  uid: 12c2f3ca-b459-4180-a12e-2b2d4b6d5aa8
- acq_time: 5
  exposure: !!python/object/apply:numpy.core.multiarray.scalar
  - *id001
  - !!binary |
    AAAAAAAAFEA=
  timestamp: 1614464054.0055833
  uid: 5bf51ebe-6439-4815-a9bb-0c824fd3afe8
- acq_time: 5
  exposure: !!python/object/apply:numpy.core.multiarray.scalar
  - *id001
  - !!binary |
    AAAAAAAAFEA=
  timestamp: 1614464263.5876548
  uid: 25d95820-b20f-4d37-901d-6c96c86462a1
- acq_time: 5
  exposure: !!python/object/apply:numpy.core.multiarray.scalar
  - *id001
  - !!binary |
    AAAAAAAAFEA=
  timestamp: 1614464433.3411603
  uid: bb969c1c-2e41-4914-91db-3440b576c752
- acq_time: 5
  exposure: !!python/object/apply:numpy.core.multiarray.scalar
  - *id001
  - !!binary |
    AAAAAAAAFEA=
  timestamp: 1614464443.6444564
  uid: 319ccd2f-7483-459c-96ac-e308b150006e
_exclude_dir:
- /mnt/data/bnl/xpdacq_special/data/user_data
- /mnt/data/bnl/xpdacq_special/data/xpdConfig
- /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml
_export_tar_dir:
- /mnt/data/bnl/xpdacq_special/data/user_data/config_base
- /mnt/data/bnl/xpdacq_special/data/user_data/userScripts
all_folders: &id002
- /mnt/data/bnl/xpdacq_special/data/user_data
- /mnt/data/bnl/xpdacq_special/data/xpdConfig
- /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml
- /mnt/data/bnl/xpdacq_special/data/user_data/config_base
- /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml/samples
- /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml/scanplans
- /mnt/data/bnl/xpdacq_special/data/user_data/tiff_base
- /mnt/data/bnl/xpdacq_special/data/user_data/userScripts
- /mnt/data/bnl/xpdacq_special/data/user_data/Import
- /mnt/data/bnl/xpdacq_special/data/user_data/userAnalysis
allfolders: *id002
archive_base_dir: /mnt/data/bnl/xpdacq_special/archive/.userbeamtimearchive
archive_base_dir_name: .userbeamtimearchive
archive_dir: /mnt/data/bnl/xpdacq_special/archive/.userbeamtimearchive/2021
archive_root_dir: /mnt/data/bnl/xpdacq_special/archive
auto_dark: true
auto_load_calib: true
base: /mnt/data/bnl/xpdacq_special/data
base_dir: /mnt/data/bnl/xpdacq_special/data
beamline_host_name:
- jupiter
beamline_id: 28-ID-1
blconfig_dir: /mnt/data/bnl/xpdacq_special/data/xpdConfig
blconfig_dir_name: xpdConfig
blconfig_name: xpd_beamline_config.yml
blconfig_path: /mnt/data/bnl/xpdacq_special/data/xpdConfig/xpd_beamline_config.yml
bt_dir: /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml
calib_config_name: xpdAcq_calib_info.yml
config_base: /mnt/data/bnl/xpdacq_special/data/user_data/config_base
dark_field_key: sc_dk_field_uid
dark_window: 0.1
det_image_field: &id003
- pe1c_image
- pe2c_image
- dexela_image
- dexela
diffraction_dets:
- pe1c
- pe1
- pe2c
- pe2
- dexela_image
- dexela
dk_window: 0.1
exp_broker_name: pdf
exp_hash_uid: f4be92ca-79c1-49ae-9bcd-59c7a8185aa3
facility: NSLS-II
frame_acq_time: 0.1
frame_acquire_time: 0.1
glbl_yaml_name: glbl.yml
glbl_yaml_path: /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml/glbl.yml
group: XPD
home: /mnt/data/bnl/xpdacq_special/data/user_data
home_dir: /mnt/data/bnl/xpdacq_special/data/user_data
home_dir_name: user_data
image_field: pe1c_image
image_fields: *id003
import_dir: /mnt/data/bnl/xpdacq_special/data/user_data/Import
inbound_proxy_address: xf28id1-ca1:5577
is_simulation: false
mask_kwargs:
  alpha: 3.0
  bs_width: 13
  edge: 30
  lower_thresh: 0.0
  tri_offset: 13
  v_asym: 0
name: glbl
outbound_proxy_address: xf28id1-ca1:5578
owner: PDF
radiogram_dets:
- blackfly_det
radiograph_names:
- blackfly_det
sample_dir: /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml/samples
scanplan_dir: /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml/scanplans
shutter_conf:
  close: 0
  open: 1
shutter_control: true
shutter_sleep: 0
simulation: false
tiff_base: /mnt/data/bnl/xpdacq_special/data/user_data/tiff_base
user_backup_dir_name: '2021'
userscript_dir: /mnt/data/bnl/xpdacq_special/data/user_data/userScripts
usrAnalysis_dir: /mnt/data/bnl/xpdacq_special/data/user_data/userAnalysis
usrScript_dir: /mnt/data/bnl/xpdacq_special/data/user_data/userScripts
xpdconfig: /mnt/data/bnl/xpdacq_special/data/xpdConfig
yaml_dir: /mnt/data/bnl/xpdacq_special/data/user_data/config_base/yml

(sys39) /m/d/b/x/data 
✔ 12:32:52 $ cat xpdConfig/xpd_beamline_config.yml
Verification time: '2021-02-26 16:47:03'
Verified by: TAC

✔ 12:33:57 $ cat user_data/userAnalysis/Ni24.D
#!/usr/bin/env python
##############################################################################
#
# xpdacq            by Billinge Group
#                   Simon J. L. Billinge sb2896@columbia.edu
#                   (c) 2016 trustees of Columbia University in the City of
#                        New York.
#                   All rights reserved
#
# File coded by:    Soham Banerjee
#
# See AUTHORS.txt for a list of people who contributed.
# See LICENSE.txt for license information.
#
##############################################################################
# Calibrant: Nickel (Ni)
# Face centered cubic cell a=3.5239 b=3.5239 c=3.5239 alpha=90.000 beta=90.000 gamma=90.000
2.034582348620  # (1, 1, 1) 8
1.761935000000  # (2, 0, 0) 6
1.245922148450  # (2, 2, 0) 12
1.062525978290  # (3, 1, 1) 24
1.017291174310  # (2, 2, 2) 8
0.881000000000  # (4, 0, 0) 6
0.808461046160  # (3, 3, 1) 24
0.787990355271  # (4, 2, 0) 24
0.719333487797  # (4, 2, 2) 24
0.678194116208  # (3, 3, 3) 8
0.622961074225  # (5, 1, 1) 24
0.595664718733  # (4, 4, 0) 12
0.587333333333  # (5, 3, 1) 48
0.557193323722  # (4, 4, 2) 24
0.537404961852  # (6, 0, 0) 6
0.531262989146  # (6, 2, 0) 24
0.508645587156  # (5, 3, 3) 24
0.493458701611  # (6, 2, 2) 24
0.488690872874  # (4, 4, 4) 8
0.470914308250  # (5, 5, 1) 24
0.458785722296  # (7, 1, 1) 24
0.440500000000  # (6, 4, 0) 24
0.430525121912  # (6, 4, 2) 48
0.427347771314  # (5, 5, 3) 24

tacaswell commented 3 years ago

rebased on #24

mrakitin commented 3 years ago

@tacaswell, is there something missing?

[E 2021-03-19 14:15:18,504.504 nslsii.ipython  logutils:56] 'NoneType' object has no attribute 'md'
Traceback (most recent call last):
  File "/home/vsts/miniconda/envs/collection-2021-1.0/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 3418, in run_code
    exec(code_obj, self.user_global_ns, self.user_ns)
  File "<ipython-input-1-19fca0ca65e8>", line 13, in <module>
    ip.parent._exec_file(f)
  File "/home/vsts/miniconda/envs/collection-2021-1.0/lib/python3.7/site-packages/IPython/core/shellapp.py", line 380, in _exec_file
    raise_exceptions=True)
  File "/home/vsts/miniconda/envs/collection-2021-1.0/lib/python3.7/site-packages/IPython/core/interactiveshell.py", line 2742, in safe_execfile
    self.compile if shell_futures else None)
  File "/home/vsts/miniconda/envs/collection-2021-1.0/lib/python3.7/site-packages/IPython/utils/py3compat.py", line 168, in execfile
    exec(compiler(f.read(), fname, 'exec'), glob, loc)
  File "/home/vsts/work/1/s/startup/94-load.py", line 107, in <module>
    RE.beamtime = bt
  File "/home/vsts/miniconda/envs/collection-2021-1.0/lib/python3.7/site-packages/xpdacq/xpdacq.py", line 161, in beamtime
    self.md.update(bt_obj.md)
AttributeError: 'NoneType' object has no attribute 'md'
==============================================================

NSLS-II-PDF / profile_collection

Mnt prep for demo #23