search

BCDA-APS / bluesky_training

Bluesky training, including instrument package
https://bcda-aps.github.io/bluesky_training/
Other
11 stars 0 forks source link

CI is failing yet it cannot be due to changes in this PR. #216

Closed prjemian closed 1 year ago

prjemian commented 1 year ago

CI is failing yet it cannot be due to the purely documentation changes in this PR.

_Originally posted by @prjemian in https://github.com/BCDA-APS/bluesky_training/issues/214#issuecomment-1662369887_

prjemian commented 1 year ago

Here's one part:

2023-08-04T04:27:39.3542176Z [I 2023-08-03 23:27:39,353 bluesky_queueserver.manager.start_manager] RE Manager configuration:
2023-08-04T04:27:39.3542900Z     zmq_control_addr: 'tcp://*:60615'
2023-08-04T04:27:39.3543328Z     zmq_private_key: None
2023-08-04T04:27:39.3552100Z     zmq_info_addr: 'tcp://*:60625'
2023-08-04T04:27:39.3595198Z     zmq_publish_console: True
2023-08-04T04:27:39.3598002Z     redis_addr: 'localhost'
2023-08-04T04:27:39.3600557Z     keep_re: True
2023-08-04T04:27:39.3603048Z     device_max_depth: 0
2023-08-04T04:27:39.3605572Z     ignore_invalid_plans: False
2023-08-04T04:27:39.3608212Z     use_ipython_kernel: False
2023-08-04T04:27:39.3610793Z     ipython_kernel_ip: 'localhost'
2023-08-04T04:27:39.3613336Z     ipython_matplotlib: None
2023-08-04T04:27:39.3616138Z     existing_plans_and_devices_path: '/home/runner/work/bluesky_training/bluesky_training/bluesky/existing_plans_and_devices.yaml'
2023-08-04T04:27:39.3619743Z     user_group_permissions_path: '/home/runner/work/bluesky_training/bluesky_training/user_group_permissions.yaml'
2023-08-04T04:27:39.3620282Z     startup_dir: '/home/runner/work/bluesky_training/bluesky_training/bluesky'
2023-08-04T04:27:39.3622373Z     startup_module: None
2023-08-04T04:27:39.3626028Z     startup_script: None
2023-08-04T04:27:39.3626389Z     startup_profile: None
2023-08-04T04:27:39.3626611Z     ipython_dir: None
2023-08-04T04:27:39.3626829Z     demo_mode: False
2023-08-04T04:27:39.3627086Z     print_console_output: True
2023-08-04T04:27:39.3627341Z     console_logging_level: 20
2023-08-04T04:27:39.3627677Z     update_existing_plans_devices: 'ENVIRONMENT_OPEN'
2023-08-04T04:27:39.3628032Z     user_group_permissions_reload: 'ON_STARTUP'
2023-08-04T04:27:39.3628284Z     emergency_lock_key: None
2023-08-04T04:27:39.3628522Z     use_persistent_metadata: False
2023-08-04T04:27:39.3628830Z     kafka_server: '127.0.0.1:9092'
2023-08-04T04:27:39.3629041Z     kafka_topic: None
2023-08-04T04:27:39.3629263Z     zmq_data_proxy_addr: None
2023-08-04T04:27:39.3629491Z     databroker_config: None
2023-08-04T04:27:39.3629632Z 
2023-08-04T04:27:39.3630403Z [W 2023-08-03 23:27:39,354 bluesky_queueserver.manager.start_manager] The file with the list of allowed plans and devices ('/home/runner/work/bluesky_training/bluesky_training/bluesky/existing_plans_and_devices.yaml') does not exist. The manager will be started with empty list. The list will be populated after RE worker environment is opened the first time.
2023-08-04T04:27:39.3631235Z [I 2023-08-03 23:27:39,358 bluesky_queueserver.manager.manager] Starting ZMQ server at 'tcp://*:60615'
2023-08-04T04:27:39.3631755Z [I 2023-08-03 23:27:39,358 bluesky_queueserver.manager.manager] ZMQ control channels: encryption disabled
2023-08-04T04:27:39.3677371Z [I 2023-08-03 23:27:39,365 bluesky_queueserver.manager.manager] Starting RE Manager process
2023-08-04T04:27:39.3780987Z [E 2023-08-03 23:27:39,376 bluesky_queueserver.manager.manager] Error occurred while loading user permissions from file '/home/runner/work/bluesky_training/bluesky_training/user_group_permissions.yaml': Error while loading user group permissions from file '/home/runner/work/bluesky_training/bluesky_training/user_group_permissions.yaml': File '/home/runner/work/bluesky_training/bluesky_training/user_group_permissions.yaml' does not exist.
2023-08-04T04:27:39.3786185Z Traceback (most recent call last):
2023-08-04T04:27:39.3787088Z   File "/home/runner/micromamba-root/envs/anaconda-test-env-py-3.9/lib/python3.9/site-packages/bluesky_queueserver/manager/profile_ops.py", line 3559, in load_user_group_permissions
2023-08-04T04:27:39.3787860Z     raise IOError(f"File '{path_to_file}' does not exist.")
2023-08-04T04:27:39.3788508Z OSError: File '/home/runner/work/bluesky_training/bluesky_training/user_group_permissions.yaml' does not exist.
2023-08-04T04:27:39.3789386Z 
2023-08-04T04:27:39.3789666Z During handling of the above exception, another exception occurred:
2023-08-04T04:27:39.3789894Z 
2023-08-04T04:27:39.3791123Z Traceback (most recent call last):
2023-08-04T04:27:39.3791932Z   File "/home/runner/micromamba-root/envs/anaconda-test-env-py-3.9/lib/python3.9/site-packages/bluesky_queueserver/manager/manager.py", line 1308, in _load_permissions_from_disk
2023-08-04T04:27:39.3845189Z     self._user_group_permissions = load_user_group_permissions(path_ug)
2023-08-04T04:27:39.3845925Z   File "/home/runner/micromamba-root/envs/anaconda-test-env-py-3.9/lib/python3.9/site-packages/bluesky_queueserver/manager/profile_ops.py", line 3571, in load_user_group_permissions
2023-08-04T04:27:39.3846612Z     raise IOError(msg)
2023-08-04T04:27:39.3847304Z OSError: Error while loading user group permissions from file '/home/runner/work/bluesky_training/bluesky_training/user_group_permissions.yaml': File '/home/runner/work/bluesky_training/bluesky_training/user_group_permissions.yaml' does not exist.
prjemian commented 1 year ago

In the CI logs, this line looks odd:

2023-08-04T05:13:52.3868595Z     existing_plans_and_devices_path: '/home/runner/work/bluesky_training/bluesky_training/bluesky/existing_plans_and_devices.yaml'

Why bluesky_training/bluesky_training? What is the pwd?

prjemian commented 1 year ago

From the logs, it's OK

2023-08-04T05:12:12.2736904Z Working directory is '/home/runner/work/bluesky_training/bluesky_training'
prjemian commented 1 year ago

Here is a problem (wrong direectory):

[E 2023-08-04 00:34:40,626 bluesky_queueserver.manager.manager] Error occurred while loading user permissions from file '/home/runner/work/bluesky_training/bluesky_training/user_group_permissions.yaml': Error while loading user group permissions from file '/home/runner/work/bluesky_training/bluesky_training/user_group_permissions.yaml': File '/home/runner/work/bluesky_training/bluesky_training/user_group_permissions.yaml' does not exist.
prjemian commented 1 year ago

Note that https://github.com/mamba-org/provision-with-micromamba is deprecated. A new one is suggested.

prjemian commented 1 year ago

Root cause: With the bluesky-queueserver, it is more sensitive to the exact starting directory. The added pushd ./bluesky/ in the unit test workflow resolved the problem. The other changes are for upgrades to the actions and added diagnostics.