search

AllenNeuralDynamics / dynamic-foraging-task

Bonsai/Harp workflow for Dynamic Foraging with Python GUI for visualization and control
MIT License
5 stars 4 forks source link

trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1 #717

Open alexpiet opened 1 month ago

alexpiet commented 1 month ago

This issue was automatically generated from GUI logs.

file: Foraging.py
function: _StartTrialLoop
line 4105
logfile: W10DT714086/W10DT714086-C_gui_log_2024-09-26_08-51-48.txt
boxes: 1A, 1C, 2A, 2B, 3D, ephys1, ephys3
count: 8
text: trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1


Log context

  RewardedHistoryF[i+1-kernel_size]=np.nanmean(RewardedHistoryT[i+1-kernel_size:i+1])

10:26:52 AM:INFO:Foraging:Foraging.py:_Save:line 2330:Saving current session, ForceSave=1
10:26:52 AM:INFO:Foraging:Foraging.py:_GetWaterCalibration:line 1009:Loaded Water Calibration
10:26:52 AM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 91:processing:C:\behavior_data\447-1-C\750108\behavior_750108_2024-09-26_09-11-57
10:26:52 AM:INFO:GenerateMetadata:GenerateMetadata.py:_handle_edge_cases:line 399:Cannot log reward delivery in session metadata - missing fields
10:26:52 AM:INFO:GenerateMetadata:GenerateMetadata.py:_get_reward_delivery:line 1234:No reward delivery metadata found!
10:26:52 AM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 906:No ephys data stream detected!
10:26:52 AM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 625:No photometry data stream detected!
10:26:52 AM:INFO:GenerateMetadata:GenerateMetadata.py:_get_high_speed_camera_stream:line 603:No camera data stream detected!
10:26:52 AM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 118:Session metadata generated successfully: True
10:26:52 AM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 119:Rig metadata generated successfully: True
10:26:52 AM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 120:Data description generated successfully: True
10:27:01 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/DelayStartTime', 22443830.912, '127.0.0.1', 4022]
10:27:01 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 22443830.913504, '127.0.0.1', 4022]
10:27:02 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTime', 22443831.412, '127.0.0.1', 4022]
10:27:02 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 22443831.413504, '127.0.0.1', 4022]
10:27:02 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTimeSoundCard', 22443831.414208, 4, '127.0.0.1']
10:27:02 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/AutoLeftWaterStartTime', 22443831.413984, '127.0.0.1', 4023]
10:27:02 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftRewardDeliveryTimeHarp', 22443831.415488, '127.0.0.1', 4023]
10:27:02 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 22443831.943712, '127.0.0.1', 4023]
10:27:02 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcomeTime', 22443831.942976, '127.0.0.1', 4022]
10:27:02 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcome', 'ErrorRight', '127.0.0.1', 4022]
10:27:03 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialEndTime', 22443832.928992, '127.0.0.1', 4022]
10:27:03 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 22443832.930496, '127.0.0.1', 4022]
10:27:03 AM:INFO:uncoupled_block:uncoupled_block.py:auto_shape_perseverance:line 137:persev at side = R, added 4 trials to both sides
10:27:42 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 22443872.16672, '127.0.0.1', 4023]
10:27:42 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 22443872.317696, '127.0.0.1', 4023]
10:27:43 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 22443872.45472, '127.0.0.1', 4023]
10:32:32 AM:ERROR:Foraging:Foraging.py:_StartTrialLoop:line 4105:trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1
alexpiet commented 1 month ago

This issue reoccurred:

file: Foraging.py
function: _StartTrialLoop
line 4264
logfile: W10DT713883/W10DT713883-A_gui_log_2024-10-14_11-16-35.txt
boxes: ephys3
count: 1
text: trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1


Log context

05:15:11 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2935417 fps=500 q=25.0 size=36000605KiB time=01:37:50.83 bitrate=50234.3kbits/s speed=   1x
05:15:12 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2935591 fps=500 q=23.0 size=36004127KiB time=01:37:51.18 bitrate=50236.2kbits/s speed=   1x
05:15:12 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2935593 fps=500 q=25.0 size=36004169KiB time=01:37:51.18 bitrate=50236.2kbits/s speed=   1x
05:15:12 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2935680 fps=500 q=23.0 size=36001944KiB time=01:37:51.36 bitrate=50231.6kbits/s speed=   1x
05:15:12 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2935680 fps=500 q=24.0 size=36003421KiB time=01:37:51.36 bitrate=50233.7kbits/s speed=   1x
05:15:12 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2935852 fps=500 q=23.0 size=36006473KiB time=01:37:51.70 bitrate=50235.0kbits/s speed=   1x
05:15:12 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2935851 fps=500 q=22.0 size=36006431KiB time=01:37:51.70 bitrate=50235.0kbits/s speed=   1x
05:15:12 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2935938 fps=500 q=24.0 size=36005981KiB time=01:37:51.87 bitrate=50232.8kbits/s speed=   1x
05:15:12 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2935938 fps=500 q=23.0 size=36004760KiB time=01:37:51.87 bitrate=50231.1kbits/s speed=   1x
05:15:13 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2936112 fps=500 q=24.0 size=36009033KiB time=01:37:52.22 bitrate=50234.1kbits/s speed=   1x
05:15:13 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2936111 fps=500 q=22.0 size=36008991KiB time=01:37:52.22 bitrate=50234.1kbits/s speed=   1x
05:15:13 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2936196 fps=500 q=23.0 size=36007832KiB time=01:37:52.39 bitrate=50231.0kbits/s speed=   1x
05:15:13 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2936197 fps=500 q=24.0 size=36008797KiB time=01:37:52.39 bitrate=50232.3kbits/s speed=   1x
05:15:13 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2936364 fps=500 q=22.0 size=36011551KiB time=01:37:52.72 bitrate=50233.3kbits/s speed=   1x
05:15:13 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2936367 fps=500 q=25.0 size=36011593KiB time=01:37:52.73 bitrate=50233.3kbits/s speed=   1x
05:15:13 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2936453 fps=500 q=23.0 size=36010904KiB time=01:37:52.90 bitrate=50230.9kbits/s speed=   1x
05:15:13 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2936453 fps=500 q=24.0 size=36011613KiB time=01:37:52.90 bitrate=50231.9kbits/s speed=   1x
05:15:14 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2936625 fps=500 q=24.0 size=36014409KiB time=01:37:53.25 bitrate=50232.8kbits/s speed=   1x
05:15:14 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2936624 fps=500 q=22.0 size=36014367KiB time=01:37:53.24 bitrate=50232.8kbits/s speed=   1x
05:15:14 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2936712 fps=500 q=23.0 size=36013976KiB time=01:37:53.42 bitrate=50230.7kbits/s speed=   1x
05:15:14 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2936712 fps=500 q=24.0 size=36014429KiB time=01:37:53.42 bitrate=50231.4kbits/s speed=   1x
05:15:14 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2936881 fps=500 q=22.0 size=36016927KiB time=01:37:53.76 bitrate=50232.0kbits/s speed=   1x
05:15:14 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2936882 fps=500 q=23.0 size=36016969KiB time=01:37:53.76 bitrate=50232.0kbits/s speed=   1x
05:15:14 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2936968 fps=500 q=25.0 size=36017245KiB time=01:37:53.93 bitrate=50230.9kbits/s speed=   1x
05:15:14 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2936967 fps=500 q=23.0 size=36017048KiB time=01:37:53.93 bitrate=50230.7kbits/s speed=   1x
05:15:15 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2937139 fps=500 q=26.0 size=36020297KiB time=01:37:54.27 bitrate=50232.3kbits/s speed=   1x
05:15:15 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2937138 fps=500 q=23.0 size=36020511KiB time=01:37:54.27 bitrate=50232.6kbits/s speed=   1x
05:15:15 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2937225 fps=500 q=24.0 size=36020376KiB time=01:37:54.45 bitrate=50230.9kbits/s speed=   1x
05:15:15 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4759:BONSAI: frame=2937225 fps=500 q=26.0 size=36021085KiB time=01:37:54.45 bitrate=50231.9kbits/s speed=   1x
05:15:15 PM:ERROR:Foraging:Foraging.py:_StartTrialLoop:line 4264:trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1
alexpiet commented 1 month ago

This issue reoccurred:

file: Foraging.py
function: _StartTrialLoop
line 4264
logfile: W10DT714027/W10DT714027-D_gui_log_2024-10-16_11-54-31.txt
boxes: 2D, ephys1
count: 2
text: trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1


Log context

01:52:03 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 13492122.89968, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:04 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 13492123.61504, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:05 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 13492124.949664, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:06 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 13492125.665024, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:07 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 13492126.949696, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:08 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 13492127.665024, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:09 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 13492128.99968, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:10 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 13492129.665024, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:11 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 13492130.99968, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:12 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 13492131.665024, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:13 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 13492133.049664, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:14 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 13492133.664992, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:15 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 13492135.049664, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:16 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 13492135.715008, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:17 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 13492137.099648, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:18 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 13492137.715008, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:19 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 13492139.149664, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:20 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 13492139.765024, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:21 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 13492141.149664, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:22 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 13492141.765024, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:23 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 13492143.149664, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:24 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 13492143.815008, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:25 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 13492145.149664, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:26 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 13492145.815008, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:27 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 13492147.199616, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:28 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 13492147.864992, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 13492149.249632, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 13492149.864992, '127.0.0.1', 4033], displaying at 0.5 Hz
01:52:31 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 13492150.93792, '127.0.0.1', 4033]
01:52:31 PM:ERROR:Foraging:Foraging.py:_StartTrialLoop:line 4264:trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1
alexpiet commented 1 month ago

This issue reoccurred:

file: Foraging.py
function: _StartTrialLoop
line 4264
logfile: W10DT714027/W10DT714027-D_gui_log_2024-10-21_09-11-19.txt
boxes: 2D
count: 1
text: trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1


Log context

10:50:03 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 13913202.127936, '127.0.0.1', 4033]
10:50:22 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 13913221.624352, '127.0.0.1', 4033]
10:50:22 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 13913221.86336, '127.0.0.1', 4033]
10:51:13 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 13913272.215712, '127.0.0.1', 4033]
10:51:13 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 13913272.377824, '127.0.0.1', 4033]
10:51:13 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 13913272.520672, '127.0.0.1', 4033]
10:51:31 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 13913290.977152, '127.0.0.1', 4033]
10:51:32 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 13913291.128448, '127.0.0.1', 4033]
10:51:40 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 13913299.63936, '127.0.0.1', 4033]
10:51:40 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 13913299.642112, '127.0.0.1', 4033]
10:51:40 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 13913299.652448, '127.0.0.1', 4033]
10:51:40 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 13913300.070112, '127.0.0.1', 4033]
10:51:41 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 13913300.36672, '127.0.0.1', 4033]
10:51:41 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 13913300.399552, '127.0.0.1', 4033]
10:51:41 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 13913300.555328, '127.0.0.1', 4033]
10:51:41 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 13913300.7672, '127.0.0.1', 4033]
10:51:42 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 13913301.417088, '127.0.0.1', 4033]
10:51:42 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 13913301.417568, '127.0.0.1', 4033]
10:51:42 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 13913302.006528, '127.0.0.1', 4033]
10:51:43 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 13913302.142368, '127.0.0.1', 4033]
10:51:47 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 13913306.83712, '127.0.0.1', 4033]
10:51:56 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 13913315.160352, '127.0.0.1', 4033]
10:51:56 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 13913315.318112, '127.0.0.1', 4033]
10:51:56 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 13913315.56144, '127.0.0.1', 4033]
10:52:08 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 13913327.8832, '127.0.0.1', 4033]
10:52:08 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 13913328.10368, '127.0.0.1', 4033]
10:52:09 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 13913328.283808, '127.0.0.1', 4033]
10:52:09 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 13913328.499744, '127.0.0.1', 4033]
10:54:06 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 13913446.073312, '127.0.0.1', 4033]
10:55:41 AM:ERROR:Foraging:Foraging.py:_StartTrialLoop:line 4264:trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1
alexpiet commented 4 weeks ago

This issue reoccurred:

file: Foraging.py
function: _StartTrialLoop
line 4277
logfile: W10DT714084/W10DT714084-B_gui_log_2024-10-25_10-09-23.txt
boxes: 1C, 2B, 2D, 3A, 3C
count: 5
text: trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1


Log context

11:37:48 AM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 908:No ephys data stream detected!
11:37:48 AM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 627:No photometry data stream detected!
11:37:48 AM:INFO:GenerateMetadata:GenerateMetadata.py:_get_high_speed_camera_stream:line 605:No camera data stream detected!
11:37:48 AM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 117:Session metadata generated successfully: True
11:37:48 AM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 118:Rig metadata generated successfully: True
11:37:48 AM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 119:Data description generated successfully: True
11:37:50 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/DelayStartTime', 17635382.726976, '127.0.0.1', 4012]
11:37:50 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 17635382.729504, '127.0.0.1', 4012]
11:37:50 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTime', 17635382.760992, '127.0.0.1', 4012]
11:37:50 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 17635382.762496, '127.0.0.1', 4012]
11:37:50 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTimeSoundCard', 17635382.7632, 4, '127.0.0.1']
11:37:53 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 17635385.339008, '127.0.0.1', 4013]
11:37:53 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcomeTime', 17635385.337984, '127.0.0.1', 4012]
11:37:53 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcome', 'ErrorLeft', '127.0.0.1', 4012]
11:37:54 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 17635386.023968, '127.0.0.1', 4013]
11:37:54 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialEndTime', 17635386.326976, '127.0.0.1', 4012]
11:37:54 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 17635386.329504, '127.0.0.1', 4012]
11:41:40 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 17635612.101056, '127.0.0.1', 4013]
11:41:40 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 17635612.249408, '127.0.0.1', 4013]
11:41:40 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 17635612.407328, '127.0.0.1', 4013]
11:41:40 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 17635612.495488, '127.0.0.1', 4013]
11:41:40 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 17635612.70688, '127.0.0.1', 4013]
11:41:40 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 17635612.781504, '127.0.0.1', 4013]
11:41:41 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 17635613.098336, '127.0.0.1', 4013]
11:41:41 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 17635613.296128, '127.0.0.1', 4013]
11:42:11 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 17635643.196096, '127.0.0.1', 4013]
11:43:19 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 17635711.593824, '127.0.0.1', 4013]
11:43:19 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 17635711.726944, '127.0.0.1', 4013]
11:43:19 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 17635711.775744, '127.0.0.1', 4013]
11:43:58 AM:ERROR:Foraging:Foraging.py:_StartTrialLoop:line 4277:trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1
alexpiet commented 4 weeks ago

This issue reoccurred:

file: Foraging.py
function: _StartTrialLoop
line 4277
logfile: W10DT714030/W10DT714030-C_gui_log_2024-10-25_10-09-05.txt
boxes: 1C, 2B, 2D, 3A, 3C
count: 5
text: trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1


Log context

  RewardedHistoryF[i+1-kernel_size]=np.nanmean(RewardedHistoryT[i+1-kernel_size:i+1])

11:33:44 AM:INFO:Foraging:Foraging.py:_Save:line 2451:Saving current session, ForceSave=1
11:33:44 AM:INFO:Foraging:Foraging.py:_GetWaterCalibration:line 1008:Loaded Water Calibration
11:33:44 AM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 90:processing:C:\behavior_data\447-3-C\757129\behavior_757129_2024-10-25_10-18-44
11:33:44 AM:INFO:GenerateMetadata:GenerateMetadata.py:_handle_edge_cases:line 398:Cannot log reward delivery in session metadata - missing fields
11:33:44 AM:INFO:GenerateMetadata:GenerateMetadata.py:_get_reward_delivery:line 1235:No reward delivery metadata found!
11:33:44 AM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 908:No ephys data stream detected!
11:33:44 AM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 627:No photometry data stream detected!
11:33:44 AM:INFO:GenerateMetadata:GenerateMetadata.py:_get_high_speed_camera_stream:line 605:No camera data stream detected!
11:33:44 AM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 117:Session metadata generated successfully: True
11:33:44 AM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 118:Rig metadata generated successfully: True
11:33:44 AM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 119:Data description generated successfully: True
11:33:46 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/DelayStartTime', 7259594.46, '127.0.0.1', 4022]
11:33:46 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 7259594.461504, '127.0.0.1', 4022]
11:33:47 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTime', 7259595.444, '127.0.0.1', 4022]
11:33:47 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 7259595.445504, '127.0.0.1', 4022]
11:33:47 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTimeSoundCard', 7259595.4464, 4, '127.0.0.1']
11:33:48 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcomeTime', 7259596.96, '127.0.0.1', 4022]
11:33:48 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcome', 'NoResponse', '127.0.0.1', 4022]
11:33:49 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialEndTime', 7259597.96, '127.0.0.1', 4022]
11:33:49 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 7259597.962496, '127.0.0.1', 4022]
11:35:25 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 7259693.904256, '127.0.0.1', 4023]
11:35:58 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 7259727.160224, '127.0.0.1', 4023]
11:35:59 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 7259727.321088, '127.0.0.1', 4023]
11:36:51 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 7259779.89168, '127.0.0.1', 4023]
11:38:28 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 7259876.97728, '127.0.0.1', 4023]
11:39:49 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 7259958.16576, '127.0.0.1', 4023]
11:40:42 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 7260011.015552, '127.0.0.1', 4023]
11:41:17 AM:ERROR:Foraging:Foraging.py:_StartTrialLoop:line 4277:trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1
alexpiet commented 4 weeks ago

This issue reoccurred:

file: Foraging.py
function: _StartTrialLoop
line 4277
logfile: W10DT714086/W10DT714086-C_gui_log_2024-10-27_15-05-44.txt
boxes: 1C
count: 1
text: trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1


Log context

05:47:04 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 25148630.420864, '127.0.0.1', 4023]
05:47:04 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftRewardDeliveryTime', 25148630.42, '127.0.0.1', 4023]
05:47:04 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcomeTime', 25148630.42, '127.0.0.1', 4022]
05:47:04 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/EarnedLeftWaterStartTime', 25148630.42, '127.0.0.1', 4023]
05:47:04 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcome', 'RewardLeft', '127.0.0.1', 4022]
05:47:04 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftRewardDeliveryTimeHarp', 25148630.422496, '127.0.0.1', 4023]
05:47:04 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 25148630.568768, '127.0.0.1', 4023]
05:47:04 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 25148630.70336, '127.0.0.1', 4023]
05:47:04 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 25148630.851104, '127.0.0.1', 4023]
05:47:04 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 25148631.004, '127.0.0.1', 4023]
05:47:04 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 25148631.151968, '127.0.0.1', 4023]
05:47:04 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 25148631.282944, '127.0.0.1', 4023]
05:47:05 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 25148631.408576, '127.0.0.1', 4023]
05:47:05 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 25148631.564864, '127.0.0.1', 4023]
05:47:05 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 25148631.696704, '127.0.0.1', 4023]
05:47:05 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 25148631.71008, '127.0.0.1', 4023]
05:47:05 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 25148631.863616, '127.0.0.1', 4023]
05:47:05 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 25148632.001504, '127.0.0.1', 4023]
05:47:05 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 25148632.132128, '127.0.0.1', 4023]
05:47:05 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 25148632.285984, '127.0.0.1', 4023]
05:47:06 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 25148632.435456, '127.0.0.1', 4023]
05:47:06 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 25148632.584416, '127.0.0.1', 4023]
05:47:06 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 25148632.75568, '127.0.0.1', 4023]
05:47:06 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 25148632.916128, '127.0.0.1', 4023]
05:47:07 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialEndTime', 25148633.425984, '127.0.0.1', 4022]
05:47:07 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 25148633.427488, '127.0.0.1', 4022]
05:48:12 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 25148698.444864, '127.0.0.1', 4023]
05:48:12 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 25148698.945728, '127.0.0.1', 4023]
05:51:50 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 25148916.977088, '127.0.0.1', 4023]
05:53:30 PM:ERROR:Foraging:Foraging.py:_StartTrialLoop:line 4277:trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1
alexpiet commented 4 weeks ago

This is becoming high priority

ellahiltonvano commented 3 weeks ago

@XX-Yin could you look into this issue and see if you have any ideas on what may be happening? It seems to be happening consistently in many different rigs.

XX-Yin commented 3 weeks ago

@ellahiltonvano Is it easy to send me a complete logfile?

alexpiet commented 3 weeks ago

All log files are available at /allen/programs/mindscope/workgroups/behavioral-dynamics/aind_logs/{hostname}_gui_log/

XX-Yin commented 3 weeks ago

That's good to know.

alexpiet commented 3 weeks ago

This issue reoccurred:

file: Foraging.py
function: _StartTrialLoop
line 4365
logfile: W10DT714027/W10DT714027-C_gui_log_2024-11-01_09-34-23.txt
boxes: 2A, 2B, 2C
count: 5
text: trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1


Log context

10:53:36 AM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 118:Rig metadata generated successfully: True
10:53:36 AM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 119:Data description generated successfully: True
10:53:42 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/DelayStartTime', 14863812.48, '127.0.0.1', 4022]
10:53:42 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 14863812.481504, '127.0.0.1', 4022]
10:53:43 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTime', 14863813.462976, '127.0.0.1', 4022]
10:53:43 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTimeSoundCard', 14863813.465248, 4, '127.0.0.1']
10:53:43 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 14863813.465504, '127.0.0.1', 4022]
10:53:43 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 14863813.591488, '127.0.0.1', 4023]
10:53:43 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcomeTime', 14863813.590976, '127.0.0.1', 4022]
10:53:43 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcome', 'ErrorRight', '127.0.0.1', 4022]
10:53:43 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 14863813.75856, '127.0.0.1', 4023]
10:53:43 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 14863813.985568, '127.0.0.1', 4023]
10:53:46 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialEndTime', 14863816.596992, '127.0.0.1', 4022]
10:53:46 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 14863816.598496, '127.0.0.1', 4022]
10:53:46 AM:INFO:uncoupled_block:uncoupled_block.py:auto_shape_perseverance:line 137:persev at side = R, added 4 trials to both sides
10:54:01 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 14863831.139904, '127.0.0.1', 4023]
10:54:01 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 14863831.603552, '127.0.0.1', 4023]
10:54:23 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 14863853.759488, '127.0.0.1', 4023]
10:54:39 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 14863869.981504, '127.0.0.1', 4023]
10:55:00 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 14863890.852256, '127.0.0.1', 4023]
10:55:20 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 14863910.186656, '127.0.0.1', 4023]
10:55:20 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 14863910.375712, '127.0.0.1', 4023]
10:55:20 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 14863910.574496, '127.0.0.1', 4023]
10:56:08 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 14863958.055488, '127.0.0.1', 4023]
10:56:33 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 14863983.728384, '127.0.0.1', 4023]
10:57:18 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 14864028.128768, '127.0.0.1', 4023]
10:58:14 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 14864084.392992, '127.0.0.1', 4023]
10:58:24 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 14864094.76352, '127.0.0.1', 4023]
10:59:35 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 14864165.053088, '127.0.0.1', 4023]
10:59:42 AM:ERROR:Foraging:Foraging.py:_StartTrialLoop:line 4365:trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1
XX-Yin commented 3 weeks ago

I checked the log file, all harp timestamps seem to be correct (If there is a missing harp timestamp, the GUI will stall). I speculate that it's related to to the following code running in a different thread to calculate the bias:

There are two concerns:

  1. Threading: Starting a new thread every ten trials for calculate_bias could lead to multiple active threads if the trials progress quickly and each thread isn’t finished before the next is created.
  2. If calculate_bias is computationally intensive, this may lead to resource exhaustion or synchronization issues. Consider joining the thread or managing it to ensure each bias_thread completes before a new one starts.

So,

  1. please don't start a new thread every ten trial, instead you can start a shared thread before running the _StartTrialLoop just like I did for worker1, workerLick and workerPlot.
  2. make sure to start a new run only after completing the previous one (example from _thread_complete#).

@micahwoodard @alexpiet I would recommend resolving both of these issues and seeing if this problem persists.

            # calculate bias every 10 trials
            if (GeneratedTrials.B_CurrentTrialN+1) % 10 == 0 and GeneratedTrials.B_CurrentTrialN+1 > 20:
                # correctly format data for bias indicator
                choice_history = [1 if x == 2 else int(x) for x in self.GeneratedTrials.B_AnimalResponseHistory]
                any_reward = [any(x) for x in np.column_stack(self.GeneratedTrials.B_RewardedHistory)]

                # use self.bias_n_size of trials to compute bias over or take n the first 0 to N trials
                l = len(choice_history)
                # FIXME: .6  of choice history is a little arbitrary. If the n_trial_back is close to equaling the
                #  trial count, the logistic regression can't be calculated because of an error saying
                #  'Cannot have number of splits n_splits=10 greater than the number of samples: 2'
                n_trial_back = self.bias_n_size if l > self.bias_n_size else \
                    round(len(np.array(choice_history)[~np.isnan(choice_history)])*.6)
                # add data to bias_indicator
                bias_thread = threading.Thread(target=self.bias_indicator.calculate_bias,
                                               kwargs={'time_point': self.GeneratedTrials.B_TrialStartTime[-1],
                                                       'choice_history': choice_history,
                                                       'reward_history': np.array(any_reward).astype(int),
                                                       'n_trial_back': n_trial_back,
                                                       'cv': 2})
                bias_thread.start()
XX-Yin commented 3 weeks ago

@micahwoodard And also, do we know the relationship between the time to run calculate_bias and the trial number?

micahwoodard commented 3 weeks ago

@XX-Yin I'm assuming the larger trial numbers take longer to calculate, but I am not super familiar with the repo used to calculate bias. @hanhou would be a better person to say for sure

alexpiet commented 3 weeks ago

It shouldn't scale with the trial number, since we are always computing bias over the same number of trials (except for the very beginning as it ramps up)

XX-Yin commented 3 weeks ago

It seems that the window size for calculating the bias is 500 trials (usually that's the largest trial length in the session)?

XX-Yin commented 3 weeks ago

@micahwoodard and @alexpiet
Another possible reason is the GUI stopped after exceeding the time limit (75m by default), but the stall function will keep running to get this "5m stall error information". We should stop the stall function in this edge case.

alexpiet commented 3 weeks ago

This issue reoccurred:

file: Foraging.py
function: _StartTrialLoop
line 4365
logfile: W10DT713669/W10DT713669-A_gui_log_2024-11-04_15-34-37.txt
boxes: ephys1
count: 1
text: trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1


Log context

08:31:28 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357143.779136, '127.0.0.1', 4003]
08:31:28 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357143.932672, '127.0.0.1', 4003]
08:31:28 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357144.516576, '127.0.0.1', 4003]
08:31:28 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357144.661632, '127.0.0.1', 4003]
08:31:29 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357145.539552, '127.0.0.1', 4003]
08:31:29 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357145.670624, '127.0.0.1', 4003]
08:31:31 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357147.006976, '127.0.0.1', 4003]
08:31:40 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357155.863488, '127.0.0.1', 4003]
08:31:58 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357173.957568, '127.0.0.1', 4003]
08:32:32 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357207.746592, '127.0.0.1', 4003]
08:32:32 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357207.885088, '127.0.0.1', 4003]
08:32:34 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357209.85952, '127.0.0.1', 4003]
08:32:34 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357210.044576, '127.0.0.1', 4003]
08:32:45 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357220.8832, '127.0.0.1', 4003]
08:32:52 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357228.46352, '127.0.0.1', 4003]
08:32:59 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357235.087712, '127.0.0.1', 4003]
08:34:16 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357312.547808, '127.0.0.1', 4003]
08:34:17 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357312.786528, '127.0.0.1', 4003]
08:34:31 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357326.697984, '127.0.0.1', 4003]
08:34:33 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357329.467456, '127.0.0.1', 4003]
08:34:47 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357343.695328, '127.0.0.1', 4003]
08:36:01 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357416.934112, '127.0.0.1', 4003]
08:36:01 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357417.119008, '127.0.0.1', 4003]
08:37:06 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357481.895392, '127.0.0.1', 4003]
08:37:14 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357490.2176, '127.0.0.1', 4003]
08:37:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357511.141344, '127.0.0.1', 4003]
08:37:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357511.329856, '127.0.0.1', 4003]
08:38:41 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357577.51136, '127.0.0.1', 4003]
08:39:16 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 357612.298656, '127.0.0.1', 4003]
08:39:32 PM:ERROR:Foraging:Foraging.py:_StartTrialLoop:line 4365:trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1
XX-Yin commented 3 weeks ago

Another possible reason is the GUI stopped after exceeding the time limit (75m by default), but the stall function will keep running to get this "5m stall error information". We should stop the stall function in this edge case

I submitted a new issue specific for this question. Users don't report the "5m stall", I speculate that at least some cases are related to this issue.

https://github.com/AllenNeuralDynamics/dynamic-foraging-task/issues/950

alexpiet commented 2 weeks ago

This issue reoccurred:

file: Foraging.py
function: _StartTrialLoop
line 4326
logfile: W10DT714086/W10DT714086-C_gui_log_2024-11-07_16-24-11.txt
boxes: 1C, 8A
count: 2
text: trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1

GUI Log snippet (click for full log)

05:42:59 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102384.250176, '127.0.0.1', 4023]
05:43:06 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102391.809152, '127.0.0.1', 4023]
05:43:38 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102424.080736, '127.0.0.1', 4023]
05:43:42 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102428.017248, '127.0.0.1', 4023]
05:43:43 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26102428.791744, '127.0.0.1', 4023]
05:43:44 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102429.332352, '127.0.0.1', 4023]
05:43:44 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102429.552256, '127.0.0.1', 4023]
05:43:46 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102431.9128, '127.0.0.1', 4023]
05:43:49 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102434.472672, '127.0.0.1', 4023]
05:43:52 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102437.644512, '127.0.0.1', 4023]
05:44:20 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102465.298624, '127.0.0.1', 4023]
05:44:20 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102465.505184, '127.0.0.1', 4023]
05:44:20 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102465.68048, '127.0.0.1', 4023]
05:44:24 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102470.0464, '127.0.0.1', 4023]
05:45:25 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102530.514496, '127.0.0.1', 4023]
05:45:25 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102530.682144, '127.0.0.1', 4023]
05:45:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102535.404832, '127.0.0.1', 4023]
05:45:38 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102543.39184, '127.0.0.1', 4023]
05:45:54 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102559.560384, '127.0.0.1', 4023]
05:46:51 PM:ERROR:Foraging:Foraging.py:_StartTrialLoop:line 4326:trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1

 Full GUI log (first 400 lines)

04:24:11 PM:INFO:Foraging:Foraging.py:start_gui_log_file:line 4707:Starting logfile!
04:24:11 PM:INFO:Foraging:Foraging.py:log_git_hash:line 4718:Python version: 3.11.9 | packaged by Anaconda, Inc. | (main, Apr 19 2024, 16:40:41) [MSC v.1916 64 bit (AMD64)]
04:24:11 PM:INFO:Foraging:Foraging.py:log_git_hash:line 4735:Current git commit branch, hash: main, 4f4f59556389b4696a28bcbed810dec00816cf61
04:24:11 PM:INFO:Foraging:Foraging.py:log_git_hash:line 4739:Current foraging_gui version: 1.6.8
04:24:11 PM:WARNING:Foraging:Foraging.py:log_git_hash:line 4749:local repository is clean
04:24:11 PM:INFO:Foraging:Foraging.py:<module>:line 4850:Setting QApplication attributes
04:24:11 PM:INFO:Foraging:Foraging.py:<module>:line 4858:Starting QApplication and Window
04:24:11 PM:INFO:Foraging:Foraging.py:__init__:line 66:Creating Window
04:24:11 PM:INFO:Foraging:Foraging.py:_GetSettings:line 1156:Loaded settings_box file
04:24:11 PM:INFO:Foraging:Foraging.py:_GetSettings:line 1184:Loaded settings file
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (temporary_video_folder), using default: C:\Users\svc_aind_behavior\Documents\temporaryvideo
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (FIP_settings), using default: C:\Users\svc_aind_behavior\Documents\FIPSettings
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (bonsai_config_path), using default: C:\Users\svc_aind_behavior\Documents\GitHub\dynamic-foraging-task\bonsai\Bonsai.config
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (open_ephys_machine_ip_address), using default: 
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (metadata_dialog_folder), using default: C:\Users\svc_aind_behavior\Documents\ForagingSettings\metadata_dialog\
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (rig_metadata_folder), using default: C:\Users\svc_aind_behavior\Documents\ForagingSettings\rig_metadata\
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (project_info_file), using default: C:\Users\svc_aind_behavior\Documents\ForagingSettings\Project Name and Funding Source v2.csv
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (schedule_path), using default: Z:\dynamic_foraging\DynamicForagingSchedule.csv
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (lick_spout_distance_box1), using default: 5000
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (lick_spout_distance_box2), using default: 5000
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (lick_spout_distance_box3), using default: 5000
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (lick_spout_distance_box4), using default: 5000
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (name_mapper_file), using default: C:\Users\svc_aind_behavior\Documents\ForagingSettings\name_mapper.json
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (save_each_trial), using default: True
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (AutomaticUpload), using default: True
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (manifest_flag_dir), using default: C:\Users\svc_aind_behavior\Documents\aind_watchdog_service\manifest
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (auto_engage), using default: True
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (clear_figure_after_save), using default: True
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (add_default_project_name), using default: True
04:24:11 PM:INFO:Foraging:Foraging.py:_LoadSchedule:line 1082:Loaded behavior schedule
04:24:11 PM:INFO:Foraging:Foraging.py:_GetWaterCalibration:line 1046:Loaded Water Calibration
04:24:11 PM:INFO:Foraging:Foraging.py:_load_most_recent_rig_json:line 1525:Found existing rig.json: rig_447-1-C_2024-11-05_09_09_40.json
04:24:11 PM:INFO:RigJsonBuilder:RigJsonBuilder.py:build_rig_json_core:line 60:building rig json
04:24:11 PM:INFO:RigJsonBuilder:RigJsonBuilder.py:build_rig_json_core:line 624:finished building rig json
04:24:12 PM:INFO:RigJsonBuilder:RigJsonBuilder.py:build_rig_json:line 53:Using existing rig json
04:24:12 PM:INFO:Foraging:Foraging.py:_LoadUI:line 259:Using ForagingGUI.ui interface
04:24:12 PM:INFO:Foraging:Foraging.py:__init__:line 107:Setting Window title: 447-1-C
04:24:12 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1353:Trying to connect to already running Bonsai
04:24:12 PM:INFO:Foraging:Foraging.py:_ConnectOSC:line 1406:connecting to GUI and Bonsai through OSC
04:24:13 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1357:Could not connect: timed out
04:24:13 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1368:Starting Bonsai
04:24:13 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4822:BONSAI logging starting
04:24:13 PM:INFO:Foraging:Foraging.py:_ConnectOSC:line 1406:connecting to GUI and Bonsai through OSC
04:24:14 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1382:Could not connect, total waiting time 0.5 seconds: timed out
04:24:15 PM:INFO:Foraging:Foraging.py:_ConnectOSC:line 1406:connecting to GUI and Bonsai through OSC
04:24:15 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1385:Connected to Bonsai after 1.0 seconds
04:24:15 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1386:Bonsai started successfully
04:24:15 PM:INFO:Foraging:Foraging.py:_ConnectSlims:line 1263:Attempting to connect to Slims
04:24:15 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26097660.859488, '127.0.0.1', 4022]
04:24:16 PM:INFO:Foraging:Foraging.py:_ConnectSlims:line 1280:Successfully connected to Slims
04:24:16 PM:WARNING:Dialogs:Dialogs.py:_LoadCalibrationParameters:line 509:could not find water calibration parameters: C:\Users\svc_aind_behavior\Documents\ForagingSettings\WaterCalibrationPar_3.json
04:24:16 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:24:16 PM:INFO:Foraging:Foraging.py:_InitializeMotorStage:line 857:Scanning for newscale stages
04:24:16 PM:INFO:Foraging:Foraging.py:_InitializeMotorStage:line 871:found 2 newscale stages
04:24:16 PM:INFO:Foraging:Foraging.py:_InitializeMotorStage:line 895:Found the newscale stage from the settings file
04:24:16 PM:INFO:Foraging:Foraging.py:_connect_stage:line 922:Successfully connected to newscale stage: 46804
04:24:16 PM:INFO:Foraging:Foraging.py:_CheckStageConnection:line 743:Checking stage connection
04:24:16 PM:INFO:Foraging:Foraging.py:_GetPositions:line 755:Grabbing current stage position
04:24:16 PM:INFO:Foraging:Foraging.py:_load_most_recent_rig_json:line 1525:Found existing rig.json: rig_447-1-C_2024-11-05_09_09_40.json
04:24:16 PM:INFO:Foraging:Foraging.py:_ReconnectBonsai:line 953:attempting to close bonsai connection
04:24:20 PM:INFO:Foraging:Foraging.py:_ReconnectBonsai:line 961:bonsai connection closed
04:24:20 PM:INFO:Foraging:Foraging.py:_ReconnectBonsai:line 963:attempting to restart bonsai
04:24:20 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1353:Trying to connect to already running Bonsai
04:24:20 PM:INFO:Foraging:Foraging.py:_ConnectOSC:line 1406:connecting to GUI and Bonsai through OSC
04:24:20 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1360:Connected to already running Bonsai
04:24:20 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1361:Bonsai started successfully
04:24:20 PM:INFO:Foraging:Foraging.py:__init__:line 203:Start up complete
04:24:21 PM:INFO:aws_util:aws_util.py:import_df_from_s3:line 115:Dataframe imported from s3://aind-behavior-data/foraging_nwb_bonsai_processed/df_sessions.pkl, len(df) = 5243
04:24:22 PM:INFO:aws_util:aws_util.py:import_df_from_s3:line 115:Dataframe imported from s3://aind-behavior-data/foraging_auto_training/df_manager_447_demo.pkl, len(df) = 3814
04:24:23 PM:INFO:aws_util:aws_util.py:download_dir_from_s3:line 140:134 objects downloaded from s3://aind-behavior-data/foraging_auto_training/saved_curriculums/ to C:\Users\svc_aind_behavior/capsule/scratch/saved_curriculums/
04:24:23 PM:INFO:curriculum_manager:curriculum_manager.py:download_curriculums:line 162:Found 29 curriculums in C:\Users\svc_aind_behavior/capsule/scratch/saved_curriculums/
04:24:23 PM:INFO:aws_util:aws_util.py:download_dir_from_s3:line 140:134 objects downloaded from s3://aind-behavior-data/foraging_auto_training/saved_curriculums/ to C:\Users\svc_aind_behavior/.aind_auto_train/curriculum_manager/
04:24:23 PM:INFO:curriculum_manager:curriculum_manager.py:download_curriculums:line 162:Found 29 curriculums in C:\Users\svc_aind_behavior/.aind_auto_train/curriculum_manager/
04:24:23 PM:INFO:Dialogs:Dialogs.py:update_auto_train_fields:line 2397:No entry found in df_training_manager for subject_id: 0
04:24:23 PM:INFO:Dialogs:Dialogs.py:_override_curriculum_clicked:line 2745:Override stage clicked: state=2
04:24:23 PM:INFO:Dialogs:Dialogs.py:_update_stage_to_apply:line 2771:Current stage to apply: unknown training stage @unknown curriculum
04:24:39 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ManualLeftWaterStartTime', 26097684.268992, '127.0.0.1', 4023]
04:24:39 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftRewardDeliveryTimeHarp', 26097684.271488, '127.0.0.1', 4023]
04:24:40 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ManualLeftWaterStartTime', 26097685.229984, '127.0.0.1', 4023]
04:24:40 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftRewardDeliveryTimeHarp', 26097685.23248, '127.0.0.1', 4023]
04:24:40 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ManualRightWaterStartTime', 26097685.861984, '127.0.0.1', 4023]
04:24:40 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightRewardDeliveryTimeHarp', 26097685.86448, '127.0.0.1', 4023]
04:24:42 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ManualRightWaterStartTime', 26097687.228, '127.0.0.1', 4023]
04:24:42 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightRewardDeliveryTimeHarp', 26097687.229504, '127.0.0.1', 4023]
04:25:26 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097731.453248, '127.0.0.1', 4023]
04:25:26 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097731.895264, '127.0.0.1', 4023]
04:25:26 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097731.957312, '127.0.0.1', 4023]
04:25:26 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097731.957344, '127.0.0.1', 4023]
04:25:26 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097732.048736, '127.0.0.1', 4023]
04:25:26 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097732.048768, '127.0.0.1', 4023]
04:25:26 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097732.074176, '127.0.0.1', 4023]
04:25:27 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097732.191584, '127.0.0.1', 4023]
04:25:27 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097732.191616, '127.0.0.1', 4023]
04:25:27 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097732.208512, '127.0.0.1', 4023]
04:25:27 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097732.472096, '127.0.0.1', 4023]
04:25:27 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097732.492224, '127.0.0.1', 4023]
04:25:27 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097732.626848, '127.0.0.1', 4023]
04:25:27 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097732.780224, '127.0.0.1', 4023]
04:25:27 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097732.941856, '127.0.0.1', 4023]
04:25:27 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097733.090464, '127.0.0.1', 4023]
04:25:27 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097733.123104, '127.0.0.1', 4023]
04:25:28 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097733.23168, '127.0.0.1', 4023]
04:25:28 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097733.40464, '127.0.0.1', 4023]
04:25:28 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097733.599616, '127.0.0.1', 4023]
04:25:28 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097733.728352, '127.0.0.1', 4023]
04:25:28 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097733.882112, '127.0.0.1', 4023]
04:25:28 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097734.0272, '127.0.0.1', 4023]
04:25:28 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097734.179936, '127.0.0.1', 4023]
04:25:29 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097734.352896, '127.0.0.1', 4023]
04:25:29 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097734.51008, '127.0.0.1', 4023]
04:25:29 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097734.67008, '127.0.0.1', 4023]
04:25:29 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097734.83952, '127.0.0.1', 4023]
04:25:29 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097734.997312, '127.0.0.1', 4023]
04:25:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097735.34592, '127.0.0.1', 4023]
04:25:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097735.725792, '127.0.0.1', 4023]
04:25:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097735.888096, '127.0.0.1', 4023]
04:25:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097736.069696, '127.0.0.1', 4023]
04:25:31 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097736.225856, '127.0.0.1', 4023]
04:25:32 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097737.211264, '127.0.0.1', 4023]
04:25:32 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097737.37088, '127.0.0.1', 4023]
04:25:32 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097737.723968, '127.0.0.1', 4023]
04:25:32 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097737.924576, '127.0.0.1', 4023]
04:25:32 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097738.118176, '127.0.0.1', 4023]
04:25:33 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097738.310336, '127.0.0.1', 4023]
04:25:33 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097738.505696, '127.0.0.1', 4023]
04:25:33 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097738.693792, '127.0.0.1', 4023]
04:25:33 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097738.881216, '127.0.0.1', 4023]
04:25:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097740.207936, '127.0.0.1', 4023]
04:25:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097740.345312, '127.0.0.1', 4023]
04:25:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097740.507424, '127.0.0.1', 4023]
04:25:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097740.682464, '127.0.0.1', 4023]
04:25:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097740.85312, '127.0.0.1', 4023]
04:25:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097741.025536, '127.0.0.1', 4023]
04:25:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097741.389184, '127.0.0.1', 4023]
04:25:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097741.60016, '127.0.0.1', 4023]
04:25:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097741.797248, '127.0.0.1', 4023]
04:25:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097741.984736, '127.0.0.1', 4023]
04:25:36 PM:INFO:Foraging:Foraging.py:_StopCurrentSession:line 3632:Stopping current trials
04:25:38 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097743.454496, '127.0.0.1', 4023]
04:25:38 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097743.643232, '127.0.0.1', 4023]
04:25:40 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097745.887296, '127.0.0.1', 4023]
04:25:40 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097746.036512, '127.0.0.1', 4023]
04:25:41 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097746.389568, '127.0.0.1', 4023]
04:25:41 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097746.67664, '127.0.0.1', 4023]
04:25:41 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097746.860192, '127.0.0.1', 4023]
04:25:42 PM:INFO:Foraging:Foraging.py:_Open:line 2974:Quick load success: C:\behavior_data\447-1-C\764788\behavior_764788_2024-11-06_16-13-38\behavior\764788_2024-11-06_16-13-38.json
04:25:42 PM:INFO:Foraging:Foraging.py:_NewSession:line 3538:New Session pressed
04:25:42 PM:INFO:Foraging:Foraging.py:_NewSession:line 3570:Resetting session run flag
04:25:42 PM:INFO:Foraging:Foraging.py:_NewSession:line 3612:New Session complete
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: DelayMin, 0.0 -> 1.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: InitiallyInactiveN, 2.0 -> 0.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMinReward, 0.0 -> 2.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMax, 60.0 -> 1.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMin, 20.0 -> 1.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RightValue, 0.03 -> 0.035
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: LeftValue, 0.03 -> 0.036
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: ID, 0.0 -> 764788.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: DelayMin, 0.0 -> 1.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: InitiallyInactiveN, 2.0 -> 0.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMinReward, 0.0 -> 2.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMax, 60.0 -> 1.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMin, 20.0 -> 1.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RightValue, 0.03 -> 0.035
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: LeftValue, 0.03 -> 0.036
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: GiveWaterL, 0.03 -> 0.036
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: GiveWaterR, 0.03 -> 0.035
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: PositionZ,  -> 5622.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: PositionY,  -> 11000.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: PositionX,  -> 7438.5
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:25:43 PM:WARNING:warnings:warnings.py:_showwarnmsg:line 110:C:\Users\svc_aind_behavior\Documents\GitHub\dynamic-foraging-task\src\foraging_gui\Visualization.py:160: RuntimeWarning: Mean of empty slice
  RewardedHistoryF[i+1-self.kernel_size]=np.nanmean(RewardedHistoryT[i+1-self.kernel_size:i+1])

04:25:43 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097749.100672, '127.0.0.1', 4023]
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: ITIMax, 8.0 -> 10.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: ITIBeta, 2.0 -> 3.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: Ignored, 100.0 -> 7.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockBeta, 20.0 -> 10.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: StopIgnores, 20.0 -> 25.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: MaxTime, 120.0 -> 75.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RewardConsumeTime, 3.0 -> 1.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: Multiplier, 0.8 -> 0.5
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RewardFamily, 1.0 -> 3.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RightValue_volume, 3.0 -> 2.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RightValue, 0.035 -> 0.024
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: LeftValue, 0.036 -> 0.026
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: Unrewarded, 200.0 -> 7.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: LeftValue_volume, 3.0 -> 2.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: ITIMax, 8.0 -> 10.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: ITIBeta, 2.0 -> 3.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: Ignored, 100.0 -> 7.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockBeta, 20.0 -> 10.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: StopIgnores, 20.0 -> 25.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: MaxTime, 120.0 -> 75.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RewardConsumeTime, 3.0 -> 1.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: Multiplier, 0.8 -> 0.5
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RewardFamily, 1.0 -> 3.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RightValue_volume, 3.0 -> 2.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RightValue, 0.035 -> 0.024
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: LeftValue, 0.036 -> 0.026
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: Unrewarded, 200.0 -> 7.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: LeftValue_volume, 3.0 -> 2.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: PositionZ, 5622.0 -> 5409.5
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: PositionY, 11000.0 -> 11002.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: PositionX, 7438.5 -> 7833.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: SuggestedWater,  -> 1.055
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BaseWeight,  -> 26.77
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: TargetWeight,  -> 22.754
04:25:44 PM:INFO:Dialogs:Dialogs.py:update_auto_train_lock:line 2949:AutoTrain disengaged!
04:25:44 PM:INFO:curriculum_manager:curriculum_manager.py:get_curriculum:line 131:Loaded a DynamicForagingCurriculum model from 'Uncoupled Baiting_curriculum_v2.3_schema_v1.0.json'.
04:25:44 PM:INFO:Dialogs:Dialogs.py:_override_curriculum_clicked:line 2745:Override stage clicked: state=0
04:25:44 PM:INFO:Dialogs:Dialogs.py:_update_curriculum_diagrams:line 2691:Selected curriculum: {'curriculum_name': 'Uncoupled Baiting', 'curriculum_version': '2.3', 'curriculum_schema_version': '1.0', 'curriculum_description': '2024-08-16 max_len = 75 mins; decrease finished trial criterion'}
04:25:44 PM:INFO:curriculum_manager:curriculum_manager.py:get_curriculum:line 131:Loaded a DynamicForagingCurriculum model from 'Uncoupled Baiting_curriculum_v2.3_schema_v1.0.json'.
04:25:44 PM:INFO:Dialogs:Dialogs.py:_update_stage_to_apply:line 2771:Current stage to apply: STAGE_2 @Uncoupled Baiting (v2.3@1.0)
04:25:44 PM:INFO:Dialogs:Dialogs.py:_update_stage_to_apply:line 2771:Current stage to apply: STAGE_2 @Uncoupled Baiting (v2.3@1.0)
04:25:44 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097749.341728, '127.0.0.1', 4023]
04:25:44 PM:INFO:Dialogs:Dialogs.py:_update_curriculum_diagrams:line 2691:Selected curriculum: {'curriculum_name': 'Uncoupled Baiting', 'curriculum_version': '2.3', 'curriculum_schema_version': '1.0', 'curriculum_description': '2024-08-16 max_len = 75 mins; decrease finished trial criterion'}
04:25:44 PM:INFO:curriculum_manager:curriculum_manager.py:get_curriculum:line 131:Loaded a DynamicForagingCurriculum model from 'Uncoupled Baiting_curriculum_v2.3_schema_v1.0.json'.
04:25:44 PM:INFO:Dialogs:Dialogs.py:update_auto_train_lock:line 2898:AutoTrain engaged! STAGE_2 @ Uncoupled Baiting (v2.3@1.0)
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3029: Widget "training_stage" not found. skipped...
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3021:Task is set to Coupled Baiting
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3029: Widget "task_url" not found. skipped...
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3029: Widget "task_schema_version" not found. skipped...
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3029: Widget "description" not found. skipped...
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:BaseRewardSum is set to 0.8
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:RewardFamily is set to 1
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:RewardPairsN is set to 1
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3041: Widget "UncoupledReward" has been disabled by the GUI. skipped...
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:Randomness is set to Exponential
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:BlockMin is set to 20
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:BlockMax is set to 35
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:BlockBeta is set to 10
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:BlockMinReward is set to 0
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:DelayMin is set to 0.3
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:DelayMax is set to 0.3
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:DelayBeta is set to 0.0
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:RewardDelay is set to 0.0
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:AutoReward is set to True
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:AutoWaterType is set to Natural
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:Multiplier is set to 0.5
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:Unrewarded is set to 7
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:Ignored is set to 7
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:ITIMin is set to 1.0
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:ITIMax is set to 10.0
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:ITIBeta is set to 3.0
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3041: Widget "ITIIncrease" has been disabled by the GUI. skipped...
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:ResponseTime is set to 3.0
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:RewardConsumeTime is set to 1.0
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:StopIgnores is set to 25
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: DelayMin, 1.0 -> 0.3
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: DelayBeta, 1.0 -> 0.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: DelayMax, 1.0 -> 0.3
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMinReward, 2.0 -> 0.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMax, 1.0 -> 35.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: ResponseTime, 1.0 -> 3.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMin, 1.0 -> 20.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RewardFamily, 3.0 -> 1.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: DelayMin, 1.0 -> 0.3
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: DelayBeta, 1.0 -> 0.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: DelayMax, 1.0 -> 0.3
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMinReward, 2.0 -> 0.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMax, 1.0 -> 35.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: ResponseTime, 1.0 -> 3.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMin, 1.0 -> 20.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RewardFamily, 3.0 -> 1.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:AdvancedBlockAuto is set to now
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:SwitchThr is set to 0.5
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:PointsInARow is set to 5
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:MaxTrial is set to 1000
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:MaxTime is set to 75
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:RightValue_volume is set to 2.0
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:LeftValue_volume is set to 2.0
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3041: Widget "warm_min_trial" has been disabled by the GUI. skipped...
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3041: Widget "warm_max_choice_ratio_bias" has been disabled by the GUI. skipped...
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3041: Widget "warm_min_finish_ratio" has been disabled by the GUI. skipped...
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3041: Widget "warm_windowsize" has been disabled by the GUI. skipped...
04:25:44 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3076:warmup is set to off
04:25:44 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097750.165792, '127.0.0.1', 4023]
04:25:45 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:25:45 PM:INFO:Dialogs:Dialogs.py:update_auto_train_fields:line 2526:Auto engage successful for mouse 764788
04:25:47 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097752.244096, '127.0.0.1', 4023]
04:25:47 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097752.49264, '127.0.0.1', 4023]
04:25:47 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097752.925184, '127.0.0.1', 4023]
04:25:48 PM:INFO:curriculum_manager:curriculum_manager.py:get_curriculum:line 131:Loaded a DynamicForagingCurriculum model from 'Uncoupled Baiting_curriculum_v2.3_schema_v1.0.json'.
04:25:48 PM:INFO:Dialogs:Dialogs.py:_update_stage_to_apply:line 2771:Current stage to apply: STAGE_2 @Uncoupled Baiting (v2.3@1.0)
04:25:48 PM:INFO:Dialogs:Dialogs.py:_update_stage_to_apply:line 2771:Current stage to apply: STAGE_2 @Uncoupled Baiting (v2.3@1.0)
04:25:48 PM:INFO:Dialogs:Dialogs.py:_update_stage_to_apply:line 2771:Current stage to apply: STAGE_2 @Uncoupled Baiting (v2.3@1.0)
04:25:48 PM:INFO:Dialogs:Dialogs.py:_update_curriculum_diagrams:line 2691:Selected curriculum: {'curriculum_name': 'Uncoupled Baiting', 'curriculum_version': '2.3', 'curriculum_schema_version': '1.0', 'curriculum_description': '2024-08-16 max_len = 75 mins; decrease finished trial criterion'}
04:25:48 PM:INFO:curriculum_manager:curriculum_manager.py:get_curriculum:line 131:Loaded a DynamicForagingCurriculum model from 'Uncoupled Baiting_curriculum_v2.3_schema_v1.0.json'.
04:25:49 PM:INFO:Dialogs:Dialogs.py:update_auto_train_lock:line 2949:AutoTrain disengaged!
04:25:51 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097756.723072, '127.0.0.1', 4023]
04:25:52 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097757.29168, '127.0.0.1', 4023]
04:25:52 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097757.404192, '127.0.0.1', 4023]
04:25:56 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:25:56 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMax, 35.0 -> 1.0
04:25:56 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMin, 20.0 -> 1.0
04:25:56 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMax, 35.0 -> 1.0
04:25:56 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMin, 20.0 -> 1.0
04:25:56 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097761.522624, '127.0.0.1', 4023]
04:25:56 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:26:05 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097770.97392, '127.0.0.1', 4023]
04:26:09 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:26:09 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: DelayMin, 0.3 -> 1.0
04:26:09 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: DelayMax, 0.3 -> 1.0
04:26:09 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: DelayMin, 0.3 -> 1.0
04:26:09 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: DelayMax, 0.3 -> 1.0
04:26:09 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:26:10 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:26:16 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097781.470208, '127.0.0.1', 4023]
04:26:16 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097781.518016, '127.0.0.1', 4023]
04:26:21 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097786.201504, '127.0.0.1', 4023]
04:26:25 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097790.870656, '127.0.0.1', 4023]
04:26:31 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097796.252928, '127.0.0.1', 4023]
04:26:32 PM:INFO:Foraging:Foraging.py:_Start:line 3780:Start button pressed: starting trial loop
04:26:32 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:26:32 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMinReward, 0.0 -> 2.0
04:26:32 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RewardFamily, 1.0 -> 3.0
04:26:32 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMinReward, 0.0 -> 2.0
04:26:32 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RewardFamily, 1.0 -> 3.0
04:26:33 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097798.578304, '127.0.0.1', 4023]
04:26:33 PM:INFO:Foraging:Foraging.py:_Start:line 3846:Starting session, with experimenter: Xinxin Yin
04:26:33 PM:INFO:Foraging:Foraging.py:_set_default_project:line 3715:Setting Project name: Behavior Platform
04:26:33 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:26:33 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:26:33 PM:INFO:Foraging:Foraging.py:_restartlogging:line 979:Restarting logging
04:26:33 PM:INFO:Foraging:Foraging.py:_GetSaveFolder:line 2760:Created new folder: C:\behavior_data\447-1-C\764788\behavior_764788_2024-11-07_16-26-33
04:26:33 PM:INFO:Foraging:Foraging.py:_GetSaveFolder:line 2763:Created new folder: C:\behavior_data\447-1-C\764788\behavior_764788_2024-11-07_16-26-33\metadata-dir
04:26:33 PM:INFO:Foraging:Foraging.py:_GetSaveFolder:line 2766:Created new folder: C:\behavior_data\447-1-C\764788\behavior_764788_2024-11-07_16-26-33\behavior
04:26:33 PM:INFO:Foraging:Foraging.py:_GetSaveFolder:line 2769:Created new folder: C:\behavior_data\447-1-C\764788\behavior_764788_2024-11-07_16-26-33\behavior\raw.harp
04:26:33 PM:INFO:Foraging:Foraging.py:_GetSaveFolder:line 2772:Created new folder: C:\behavior_data\447-1-C\764788\behavior_764788_2024-11-07_16-26-33\behavior-videos
04:26:33 PM:INFO:Foraging:Foraging.py:_GetSaveFolder:line 2775:Created new folder: C:\behavior_data\447-1-C\764788\behavior_764788_2024-11-07_16-26-33\fib
04:26:33 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/loggerstarted', 'C:\\behavior_data\\447-1-C\\764788\\behavior_764788_2024-11-07_16-26-33\\behavior\\raw.harp', '127.0.0.1', 4022]
04:26:33 PM:INFO:Foraging:Foraging.py:log_session:line 4159:Starting log file at C:\behavior_data\447-1-C\764788\behavior_764788_2024-11-07_16-26-33\behavior
04:26:33 PM:INFO:Foraging:Foraging.py:_StartTrialLoop:line 4176:starting trial loop
04:26:33 PM:INFO:Foraging:Foraging.py:_StartTrialLoop:line 4204:Current trial: 1
04:26:33 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialStartTime', 26097799.082976, '127.0.0.1', 4022]
04:26:33 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ITIStartTimeHarp', 26097799.086496, '127.0.0.1', 4022]
04:26:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/DelayStartTime', 26097800.813984, '127.0.0.1', 4022]
04:26:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26097800.81648, '127.0.0.1', 4022]
04:26:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTime', 26097801.796, '127.0.0.1', 4022]
04:26:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26097801.799488, '127.0.0.1', 4022]
04:26:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTimeSoundCard', 26097801.799232, 4, '127.0.0.1']
04:26:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097801.919136, '127.0.0.1', 4023]
04:26:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcomeTime', 26097801.918976, '127.0.0.1', 4022]
04:26:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcome', 'ErrorRight', '127.0.0.1', 4022]
04:26:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097802.037536, '127.0.0.1', 4023]
04:26:37 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097802.426944, '127.0.0.1', 4023]
04:26:37 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097802.587264, '127.0.0.1', 4023]
04:26:37 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialEndTime', 26097802.912992, '127.0.0.1', 4022]
04:26:37 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26097802.915488, '127.0.0.1', 4022]
04:26:37 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097802.916352, '127.0.0.1', 4023]
04:26:37 PM:INFO:Foraging:Foraging.py:_StartTrialLoop:line 4204:Current trial: 2
04:26:37 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialStartTime', 26097802.966976, '127.0.0.1', 4022]
04:26:37 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ITIStartTimeHarp', 26097802.970496, '127.0.0.1', 4022]
04:26:37 PM:INFO:Foraging:Foraging.py:_Save:line 2452:Saving current session, ForceSave=1
04:26:37 PM:INFO:Foraging:Foraging.py:_GetWaterCalibration:line 1046:Loaded Water Calibration
04:26:37 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 90:processing:C:\behavior_data\447-1-C\764788\behavior_764788_2024-11-07_16-26-33
04:26:37 PM:INFO:GenerateMetadata:GenerateMetadata.py:_handle_edge_cases:line 398:Cannot log reward delivery in session metadata - missing fields
04:26:37 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_reward_delivery:line 1235:No reward delivery metadata found!
04:26:37 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 908:No ephys data stream detected!
04:26:37 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 627:No photometry data stream detected!
04:26:37 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_high_speed_camera_stream:line 605:No camera data stream detected!
04:26:37 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 117:Session metadata generated successfully: True
04:26:37 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 118:Rig metadata generated successfully: True
04:26:37 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 119:Data description generated successfully: True
04:26:39 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/DelayStartTime', 26097805.012992, '127.0.0.1', 4022]
04:26:39 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26097805.014496, '127.0.0.1', 4022]
04:26:40 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTime', 26097805.996, '127.0.0.1', 4022]
04:26:40 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26097805.997504, '127.0.0.1', 4022]
04:26:40 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTimeSoundCard', 26097805.99824, 4, '127.0.0.1']
04:26:40 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097806.119424, '127.0.0.1', 4023]
04:26:40 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcomeTime', 26097806.118976, '127.0.0.1', 4022]
04:26:40 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcome', 'ErrorRight', '127.0.0.1', 4022]
04:26:41 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097806.518528, '127.0.0.1', 4023]
04:26:41 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097806.936608, '127.0.0.1', 4023]
04:26:41 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialEndTime', 26097807.112992, '127.0.0.1', 4022]
04:26:41 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26097807.114496, '127.0.0.1', 4022]
04:26:41 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097807.170752, '127.0.0.1', 4023]
04:26:41 PM:INFO:Foraging:Foraging.py:_StartTrialLoop:line 4204:Current trial: 3
04:26:42 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialStartTime', 26097807.185984, '127.0.0.1', 4022]
04:26:42 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ITIStartTimeHarp', 26097807.189504, '127.0.0.1', 4022]
04:26:42 PM:WARNING:warnings:warnings.py:_showwarnmsg:line 110:C:\Users\svc_aind_behavior\Documents\GitHub\dynamic-foraging-task\src\foraging_gui\Visualization.py:160: RuntimeWarning: Mean of empty slice
  RewardedHistoryF[i+1-self.kernel_size]=np.nanmean(RewardedHistoryT[i+1-self.kernel_size:i+1])

04:26:42 PM:INFO:Foraging:Foraging.py:_Save:line 2452:Saving current session, ForceSave=1
04:26:42 PM:INFO:Foraging:Foraging.py:_GetWaterCalibration:line 1046:Loaded Water Calibration
04:26:42 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 90:processing:C:\behavior_data\447-1-C\764788\behavior_764788_2024-11-07_16-26-33
04:26:42 PM:INFO:GenerateMetadata:GenerateMetadata.py:_handle_edge_cases:line 398:Cannot log reward delivery in session metadata - missing fields
04:26:42 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_reward_delivery:line 1235:No reward delivery metadata found!
04:26:42 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 908:No ephys data stream detected!
04:26:42 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 627:No photometry data stream detected!
04:26:42 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_high_speed_camera_stream:line 605:No camera data stream detected!
04:26:42 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 117:Session metadata generated successfully: True

alexpiet commented 2 weeks ago

This issue reoccurred:

file: Foraging.py
function: _StartTrialLoop
line 4326
logfile: W10DT714086/W10DT714086-C_gui_log_2024-11-07_16-24-11.txt
boxes: 1C
count: 1
text: trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1

GUI Log snippet (click for full log)

05:42:59 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102384.250176, '127.0.0.1', 4023]
05:43:06 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102391.809152, '127.0.0.1', 4023]
05:43:38 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102424.080736, '127.0.0.1', 4023]
05:43:42 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102428.017248, '127.0.0.1', 4023]
05:43:43 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26102428.791744, '127.0.0.1', 4023]
05:43:44 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102429.332352, '127.0.0.1', 4023]
05:43:44 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102429.552256, '127.0.0.1', 4023]
05:43:46 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102431.9128, '127.0.0.1', 4023]
05:43:49 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102434.472672, '127.0.0.1', 4023]
05:43:52 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102437.644512, '127.0.0.1', 4023]
05:44:20 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102465.298624, '127.0.0.1', 4023]
05:44:20 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102465.505184, '127.0.0.1', 4023]
05:44:20 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102465.68048, '127.0.0.1', 4023]
05:44:24 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102470.0464, '127.0.0.1', 4023]
05:45:25 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102530.514496, '127.0.0.1', 4023]
05:45:25 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102530.682144, '127.0.0.1', 4023]
05:45:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102535.404832, '127.0.0.1', 4023]
05:45:38 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102543.39184, '127.0.0.1', 4023]
05:45:54 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102559.560384, '127.0.0.1', 4023]
05:46:51 PM:ERROR:Foraging:Foraging.py:_StartTrialLoop:line 4326:trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1

 Full GUI log (first 200 and last 200 lines)

04:24:11 PM:INFO:Foraging:Foraging.py:start_gui_log_file:line 4707:Starting logfile!
04:24:11 PM:INFO:Foraging:Foraging.py:log_git_hash:line 4718:Python version: 3.11.9 | packaged by Anaconda, Inc. | (main, Apr 19 2024, 16:40:41) [MSC v.1916 64 bit (AMD64)]
04:24:11 PM:INFO:Foraging:Foraging.py:log_git_hash:line 4735:Current git commit branch, hash: main, 4f4f59556389b4696a28bcbed810dec00816cf61
04:24:11 PM:INFO:Foraging:Foraging.py:log_git_hash:line 4739:Current foraging_gui version: 1.6.8
04:24:11 PM:WARNING:Foraging:Foraging.py:log_git_hash:line 4749:local repository is clean
04:24:11 PM:INFO:Foraging:Foraging.py:<module>:line 4850:Setting QApplication attributes
04:24:11 PM:INFO:Foraging:Foraging.py:<module>:line 4858:Starting QApplication and Window
04:24:11 PM:INFO:Foraging:Foraging.py:__init__:line 66:Creating Window
04:24:11 PM:INFO:Foraging:Foraging.py:_GetSettings:line 1156:Loaded settings_box file
04:24:11 PM:INFO:Foraging:Foraging.py:_GetSettings:line 1184:Loaded settings file
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (temporary_video_folder), using default: C:\Users\svc_aind_behavior\Documents\temporaryvideo
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (FIP_settings), using default: C:\Users\svc_aind_behavior\Documents\FIPSettings
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (bonsai_config_path), using default: C:\Users\svc_aind_behavior\Documents\GitHub\dynamic-foraging-task\bonsai\Bonsai.config
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (open_ephys_machine_ip_address), using default: 
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (metadata_dialog_folder), using default: C:\Users\svc_aind_behavior\Documents\ForagingSettings\metadata_dialog\
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (rig_metadata_folder), using default: C:\Users\svc_aind_behavior\Documents\ForagingSettings\rig_metadata\
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (project_info_file), using default: C:\Users\svc_aind_behavior\Documents\ForagingSettings\Project Name and Funding Source v2.csv
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (schedule_path), using default: Z:\dynamic_foraging\DynamicForagingSchedule.csv
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (lick_spout_distance_box1), using default: 5000
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (lick_spout_distance_box2), using default: 5000
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (lick_spout_distance_box3), using default: 5000
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (lick_spout_distance_box4), using default: 5000
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (name_mapper_file), using default: C:\Users\svc_aind_behavior\Documents\ForagingSettings\name_mapper.json
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (save_each_trial), using default: True
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (AutomaticUpload), using default: True
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (manifest_flag_dir), using default: C:\Users\svc_aind_behavior\Documents\aind_watchdog_service\manifest
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (auto_engage), using default: True
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (clear_figure_after_save), using default: True
04:24:11 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1194:Missing setting (add_default_project_name), using default: True
04:24:11 PM:INFO:Foraging:Foraging.py:_LoadSchedule:line 1082:Loaded behavior schedule
04:24:11 PM:INFO:Foraging:Foraging.py:_GetWaterCalibration:line 1046:Loaded Water Calibration
04:24:11 PM:INFO:Foraging:Foraging.py:_load_most_recent_rig_json:line 1525:Found existing rig.json: rig_447-1-C_2024-11-05_09_09_40.json
04:24:11 PM:INFO:RigJsonBuilder:RigJsonBuilder.py:build_rig_json_core:line 60:building rig json
04:24:11 PM:INFO:RigJsonBuilder:RigJsonBuilder.py:build_rig_json_core:line 624:finished building rig json
04:24:12 PM:INFO:RigJsonBuilder:RigJsonBuilder.py:build_rig_json:line 53:Using existing rig json
04:24:12 PM:INFO:Foraging:Foraging.py:_LoadUI:line 259:Using ForagingGUI.ui interface
04:24:12 PM:INFO:Foraging:Foraging.py:__init__:line 107:Setting Window title: 447-1-C
04:24:12 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1353:Trying to connect to already running Bonsai
04:24:12 PM:INFO:Foraging:Foraging.py:_ConnectOSC:line 1406:connecting to GUI and Bonsai through OSC
04:24:13 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1357:Could not connect: timed out
04:24:13 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1368:Starting Bonsai
04:24:13 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4822:BONSAI logging starting
04:24:13 PM:INFO:Foraging:Foraging.py:_ConnectOSC:line 1406:connecting to GUI and Bonsai through OSC
04:24:14 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1382:Could not connect, total waiting time 0.5 seconds: timed out
04:24:15 PM:INFO:Foraging:Foraging.py:_ConnectOSC:line 1406:connecting to GUI and Bonsai through OSC
04:24:15 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1385:Connected to Bonsai after 1.0 seconds
04:24:15 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1386:Bonsai started successfully
04:24:15 PM:INFO:Foraging:Foraging.py:_ConnectSlims:line 1263:Attempting to connect to Slims
04:24:15 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26097660.859488, '127.0.0.1', 4022]
04:24:16 PM:INFO:Foraging:Foraging.py:_ConnectSlims:line 1280:Successfully connected to Slims
04:24:16 PM:WARNING:Dialogs:Dialogs.py:_LoadCalibrationParameters:line 509:could not find water calibration parameters: C:\Users\svc_aind_behavior\Documents\ForagingSettings\WaterCalibrationPar_3.json
04:24:16 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:24:16 PM:INFO:Foraging:Foraging.py:_InitializeMotorStage:line 857:Scanning for newscale stages
04:24:16 PM:INFO:Foraging:Foraging.py:_InitializeMotorStage:line 871:found 2 newscale stages
04:24:16 PM:INFO:Foraging:Foraging.py:_InitializeMotorStage:line 895:Found the newscale stage from the settings file
04:24:16 PM:INFO:Foraging:Foraging.py:_connect_stage:line 922:Successfully connected to newscale stage: 46804
04:24:16 PM:INFO:Foraging:Foraging.py:_CheckStageConnection:line 743:Checking stage connection
04:24:16 PM:INFO:Foraging:Foraging.py:_GetPositions:line 755:Grabbing current stage position
04:24:16 PM:INFO:Foraging:Foraging.py:_load_most_recent_rig_json:line 1525:Found existing rig.json: rig_447-1-C_2024-11-05_09_09_40.json
04:24:16 PM:INFO:Foraging:Foraging.py:_ReconnectBonsai:line 953:attempting to close bonsai connection
04:24:20 PM:INFO:Foraging:Foraging.py:_ReconnectBonsai:line 961:bonsai connection closed
04:24:20 PM:INFO:Foraging:Foraging.py:_ReconnectBonsai:line 963:attempting to restart bonsai
04:24:20 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1353:Trying to connect to already running Bonsai
04:24:20 PM:INFO:Foraging:Foraging.py:_ConnectOSC:line 1406:connecting to GUI and Bonsai through OSC
04:24:20 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1360:Connected to already running Bonsai
04:24:20 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1361:Bonsai started successfully
04:24:20 PM:INFO:Foraging:Foraging.py:__init__:line 203:Start up complete
04:24:21 PM:INFO:aws_util:aws_util.py:import_df_from_s3:line 115:Dataframe imported from s3://aind-behavior-data/foraging_nwb_bonsai_processed/df_sessions.pkl, len(df) = 5243
04:24:22 PM:INFO:aws_util:aws_util.py:import_df_from_s3:line 115:Dataframe imported from s3://aind-behavior-data/foraging_auto_training/df_manager_447_demo.pkl, len(df) = 3814
04:24:23 PM:INFO:aws_util:aws_util.py:download_dir_from_s3:line 140:134 objects downloaded from s3://aind-behavior-data/foraging_auto_training/saved_curriculums/ to C:\Users\svc_aind_behavior/capsule/scratch/saved_curriculums/
04:24:23 PM:INFO:curriculum_manager:curriculum_manager.py:download_curriculums:line 162:Found 29 curriculums in C:\Users\svc_aind_behavior/capsule/scratch/saved_curriculums/
04:24:23 PM:INFO:aws_util:aws_util.py:download_dir_from_s3:line 140:134 objects downloaded from s3://aind-behavior-data/foraging_auto_training/saved_curriculums/ to C:\Users\svc_aind_behavior/.aind_auto_train/curriculum_manager/
04:24:23 PM:INFO:curriculum_manager:curriculum_manager.py:download_curriculums:line 162:Found 29 curriculums in C:\Users\svc_aind_behavior/.aind_auto_train/curriculum_manager/
04:24:23 PM:INFO:Dialogs:Dialogs.py:update_auto_train_fields:line 2397:No entry found in df_training_manager for subject_id: 0
04:24:23 PM:INFO:Dialogs:Dialogs.py:_override_curriculum_clicked:line 2745:Override stage clicked: state=2
04:24:23 PM:INFO:Dialogs:Dialogs.py:_update_stage_to_apply:line 2771:Current stage to apply: unknown training stage @unknown curriculum
04:24:39 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ManualLeftWaterStartTime', 26097684.268992, '127.0.0.1', 4023]
04:24:39 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftRewardDeliveryTimeHarp', 26097684.271488, '127.0.0.1', 4023]
04:24:40 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ManualLeftWaterStartTime', 26097685.229984, '127.0.0.1', 4023]
04:24:40 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftRewardDeliveryTimeHarp', 26097685.23248, '127.0.0.1', 4023]
04:24:40 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ManualRightWaterStartTime', 26097685.861984, '127.0.0.1', 4023]
04:24:40 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightRewardDeliveryTimeHarp', 26097685.86448, '127.0.0.1', 4023]
04:24:42 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ManualRightWaterStartTime', 26097687.228, '127.0.0.1', 4023]
04:24:42 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightRewardDeliveryTimeHarp', 26097687.229504, '127.0.0.1', 4023]
04:25:26 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097731.453248, '127.0.0.1', 4023]
04:25:26 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097731.895264, '127.0.0.1', 4023]
04:25:26 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097731.957312, '127.0.0.1', 4023]
04:25:26 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097731.957344, '127.0.0.1', 4023]
04:25:26 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097732.048736, '127.0.0.1', 4023]
04:25:26 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097732.048768, '127.0.0.1', 4023]
04:25:26 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097732.074176, '127.0.0.1', 4023]
04:25:27 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097732.191584, '127.0.0.1', 4023]
04:25:27 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097732.191616, '127.0.0.1', 4023]
04:25:27 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097732.208512, '127.0.0.1', 4023]
04:25:27 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097732.472096, '127.0.0.1', 4023]
04:25:27 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097732.492224, '127.0.0.1', 4023]
04:25:27 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097732.626848, '127.0.0.1', 4023]
04:25:27 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097732.780224, '127.0.0.1', 4023]
04:25:27 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097732.941856, '127.0.0.1', 4023]
04:25:27 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097733.090464, '127.0.0.1', 4023]
04:25:27 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097733.123104, '127.0.0.1', 4023]
04:25:28 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097733.23168, '127.0.0.1', 4023]
04:25:28 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097733.40464, '127.0.0.1', 4023]
04:25:28 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097733.599616, '127.0.0.1', 4023]
04:25:28 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097733.728352, '127.0.0.1', 4023]
04:25:28 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097733.882112, '127.0.0.1', 4023]
04:25:28 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097734.0272, '127.0.0.1', 4023]
04:25:28 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097734.179936, '127.0.0.1', 4023]
04:25:29 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097734.352896, '127.0.0.1', 4023]
04:25:29 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097734.51008, '127.0.0.1', 4023]
04:25:29 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097734.67008, '127.0.0.1', 4023]
04:25:29 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097734.83952, '127.0.0.1', 4023]
04:25:29 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097734.997312, '127.0.0.1', 4023]
04:25:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097735.34592, '127.0.0.1', 4023]
04:25:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097735.725792, '127.0.0.1', 4023]
04:25:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097735.888096, '127.0.0.1', 4023]
04:25:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097736.069696, '127.0.0.1', 4023]
04:25:31 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097736.225856, '127.0.0.1', 4023]
04:25:32 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097737.211264, '127.0.0.1', 4023]
04:25:32 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097737.37088, '127.0.0.1', 4023]
04:25:32 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097737.723968, '127.0.0.1', 4023]
04:25:32 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097737.924576, '127.0.0.1', 4023]
04:25:32 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097738.118176, '127.0.0.1', 4023]
04:25:33 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097738.310336, '127.0.0.1', 4023]
04:25:33 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097738.505696, '127.0.0.1', 4023]
04:25:33 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097738.693792, '127.0.0.1', 4023]
04:25:33 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097738.881216, '127.0.0.1', 4023]
04:25:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097740.207936, '127.0.0.1', 4023]
04:25:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097740.345312, '127.0.0.1', 4023]
04:25:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097740.507424, '127.0.0.1', 4023]
04:25:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097740.682464, '127.0.0.1', 4023]
04:25:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097740.85312, '127.0.0.1', 4023]
04:25:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097741.025536, '127.0.0.1', 4023]
04:25:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097741.389184, '127.0.0.1', 4023]
04:25:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097741.60016, '127.0.0.1', 4023]
04:25:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097741.797248, '127.0.0.1', 4023]
04:25:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097741.984736, '127.0.0.1', 4023]
04:25:36 PM:INFO:Foraging:Foraging.py:_StopCurrentSession:line 3632:Stopping current trials
04:25:38 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097743.454496, '127.0.0.1', 4023]
04:25:38 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097743.643232, '127.0.0.1', 4023]
04:25:40 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097745.887296, '127.0.0.1', 4023]
04:25:40 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097746.036512, '127.0.0.1', 4023]
04:25:41 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097746.389568, '127.0.0.1', 4023]
04:25:41 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097746.67664, '127.0.0.1', 4023]
04:25:41 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26097746.860192, '127.0.0.1', 4023]
04:25:42 PM:INFO:Foraging:Foraging.py:_Open:line 2974:Quick load success: C:\behavior_data\447-1-C\764788\behavior_764788_2024-11-06_16-13-38\behavior\764788_2024-11-06_16-13-38.json
04:25:42 PM:INFO:Foraging:Foraging.py:_NewSession:line 3538:New Session pressed
04:25:42 PM:INFO:Foraging:Foraging.py:_NewSession:line 3570:Resetting session run flag
04:25:42 PM:INFO:Foraging:Foraging.py:_NewSession:line 3612:New Session complete
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: DelayMin, 0.0 -> 1.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: InitiallyInactiveN, 2.0 -> 0.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMinReward, 0.0 -> 2.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMax, 60.0 -> 1.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMin, 20.0 -> 1.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RightValue, 0.03 -> 0.035
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: LeftValue, 0.03 -> 0.036
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: ID, 0.0 -> 764788.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: DelayMin, 0.0 -> 1.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: InitiallyInactiveN, 2.0 -> 0.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMinReward, 0.0 -> 2.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMax, 60.0 -> 1.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockMin, 20.0 -> 1.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RightValue, 0.03 -> 0.035
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: LeftValue, 0.03 -> 0.036
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: GiveWaterL, 0.03 -> 0.036
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: GiveWaterR, 0.03 -> 0.035
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: PositionZ,  -> 5622.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: PositionY,  -> 11000.0
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: PositionX,  -> 7438.5
04:25:42 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:25:43 PM:WARNING:warnings:warnings.py:_showwarnmsg:line 110:C:\Users\svc_aind_behavior\Documents\GitHub\dynamic-foraging-task\src\foraging_gui\Visualization.py:160: RuntimeWarning: Mean of empty slice
  RewardedHistoryF[i+1-self.kernel_size]=np.nanmean(RewardedHistoryT[i+1-self.kernel_size:i+1])

04:25:43 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26097749.100672, '127.0.0.1', 4023]
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1880:processing parameter changes
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: ITIMax, 8.0 -> 10.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: ITIBeta, 2.0 -> 3.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: Ignored, 100.0 -> 7.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockBeta, 20.0 -> 10.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: StopIgnores, 20.0 -> 25.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: MaxTime, 120.0 -> 75.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RewardConsumeTime, 3.0 -> 1.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: Multiplier, 0.8 -> 0.5
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RewardFamily, 1.0 -> 3.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RightValue_volume, 3.0 -> 2.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RightValue, 0.035 -> 0.024
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: LeftValue, 0.036 -> 0.026
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: Unrewarded, 200.0 -> 7.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: LeftValue_volume, 3.0 -> 2.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: ITIMax, 8.0 -> 10.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: ITIBeta, 2.0 -> 3.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: Ignored, 100.0 -> 7.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: BlockBeta, 20.0 -> 10.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: StopIgnores, 20.0 -> 25.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: MaxTime, 120.0 -> 75.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RewardConsumeTime, 3.0 -> 1.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: Multiplier, 0.8 -> 0.5
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RewardFamily, 1.0 -> 3.0
04:25:44 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1939:Changing parameter: RightValue_volume, 3.0 -> 2.0
_____SKIPPING_____
05:40:56 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTime', 26102261.197984, '127.0.0.1', 4022]
05:40:56 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26102261.20048, '127.0.0.1', 4022]
05:40:56 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTimeSoundCard', 26102261.201216, 4, '127.0.0.1']
05:40:56 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102261.308288, '127.0.0.1', 4023]
05:40:56 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcomeTime', 26102261.308, '127.0.0.1', 4022]
05:40:56 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcome', 'ErrorRight', '127.0.0.1', 4022]
05:40:57 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialEndTime', 26102262.297984, '127.0.0.1', 4022]
05:40:57 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26102262.30048, '127.0.0.1', 4022]
05:40:57 PM:INFO:Foraging:Foraging.py:_StartTrialLoop:line 4204:Current trial: 486
05:40:57 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialStartTime', 26102262.373984, '127.0.0.1', 4022]
05:40:57 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ITIStartTimeHarp', 26102262.377504, '127.0.0.1', 4022]
05:40:57 PM:WARNING:warnings:warnings.py:_showwarnmsg:line 110:C:\Users\svc_aind_behavior\Documents\GitHub\dynamic-foraging-task\src\foraging_gui\Visualization.py:160: RuntimeWarning: Mean of empty slice
  RewardedHistoryF[i+1-self.kernel_size]=np.nanmean(RewardedHistoryT[i+1-self.kernel_size:i+1])

05:40:57 PM:INFO:Foraging:Foraging.py:_Save:line 2452:Saving current session, ForceSave=1
05:40:57 PM:INFO:Foraging:Foraging.py:_GetWaterCalibration:line 1046:Loaded Water Calibration
05:40:57 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 90:processing:C:\behavior_data\447-1-C\764788\behavior_764788_2024-11-07_16-26-33
05:40:57 PM:INFO:GenerateMetadata:GenerateMetadata.py:_handle_edge_cases:line 398:Cannot log reward delivery in session metadata - missing fields
05:40:57 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_reward_delivery:line 1235:No reward delivery metadata found!
05:40:57 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 908:No ephys data stream detected!
05:40:57 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 627:No photometry data stream detected!
05:40:57 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_high_speed_camera_stream:line 605:No camera data stream detected!
05:40:57 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 117:Session metadata generated successfully: True
05:40:57 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 118:Rig metadata generated successfully: True
05:40:57 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 119:Data description generated successfully: True
05:40:59 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102265.111232, '127.0.0.1', 4023]
05:41:03 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102268.58176, '127.0.0.1', 4023]
05:41:03 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/DelayStartTime', 26102269.081984, '127.0.0.1', 4022]
05:41:03 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26102269.083488, '127.0.0.1', 4022]
05:41:04 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTime', 26102270.064992, '127.0.0.1', 4022]
05:41:04 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26102270.066496, '127.0.0.1', 4022]
05:41:04 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTimeSoundCard', 26102270.067136, 4, '127.0.0.1']
05:41:05 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26102270.346688, '127.0.0.1', 4023]
05:41:05 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcomeTime', 26102270.345984, '127.0.0.1', 4022]
05:41:05 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftRewardDeliveryTime', 26102270.345984, '127.0.0.1', 4023]
05:41:05 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/EarnedLeftWaterStartTime', 26102270.345984, '127.0.0.1', 4023]
05:41:05 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcome', 'RewardLeft', '127.0.0.1', 4022]
05:41:05 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftRewardDeliveryTimeHarp', 26102270.34848, '127.0.0.1', 4023]
05:41:06 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialEndTime', 26102271.332, '127.0.0.1', 4022]
05:41:06 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26102271.333504, '127.0.0.1', 4022]
05:41:06 PM:INFO:Foraging:Foraging.py:_StartTrialLoop:line 4204:Current trial: 487
05:41:06 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialStartTime', 26102271.412992, '127.0.0.1', 4022]
05:41:06 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ITIStartTimeHarp', 26102271.417504, '127.0.0.1', 4022]
05:41:06 PM:WARNING:warnings:warnings.py:_showwarnmsg:line 110:C:\Users\svc_aind_behavior\Documents\GitHub\dynamic-foraging-task\src\foraging_gui\Visualization.py:160: RuntimeWarning: Mean of empty slice
  RewardedHistoryF[i+1-self.kernel_size]=np.nanmean(RewardedHistoryT[i+1-self.kernel_size:i+1])

05:41:06 PM:INFO:Foraging:Foraging.py:_Save:line 2452:Saving current session, ForceSave=1
05:41:06 PM:INFO:Foraging:Foraging.py:_GetWaterCalibration:line 1046:Loaded Water Calibration
05:41:06 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 90:processing:C:\behavior_data\447-1-C\764788\behavior_764788_2024-11-07_16-26-33
05:41:06 PM:INFO:GenerateMetadata:GenerateMetadata.py:_handle_edge_cases:line 398:Cannot log reward delivery in session metadata - missing fields
05:41:06 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_reward_delivery:line 1235:No reward delivery metadata found!
05:41:06 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 908:No ephys data stream detected!
05:41:06 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 627:No photometry data stream detected!
05:41:06 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_high_speed_camera_stream:line 605:No camera data stream detected!
05:41:06 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 117:Session metadata generated successfully: True
05:41:06 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 118:Rig metadata generated successfully: True
05:41:06 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 119:Data description generated successfully: True
05:41:06 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26102272.01312, '127.0.0.1', 4023]
05:41:07 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26102272.633088, '127.0.0.1', 4023]
05:41:07 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/DelayStartTime', 26102273.064992, '127.0.0.1', 4022]
05:41:07 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26102273.066496, '127.0.0.1', 4022]
05:41:08 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTime', 26102274.048, '127.0.0.1', 4022]
05:41:08 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26102274.050496, '127.0.0.1', 4022]
05:41:08 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTimeSoundCard', 26102274.051136, 4, '127.0.0.1']
05:41:11 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcomeTime', 26102277.064992, '127.0.0.1', 4022]
05:41:11 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcome', 'NoResponse', '127.0.0.1', 4022]
05:41:12 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialEndTime', 26102278.064992, '127.0.0.1', 4022]
05:41:12 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26102278.066496, '127.0.0.1', 4022]
05:41:12 PM:INFO:Foraging:Foraging.py:_StartTrialLoop:line 4204:Current trial: 488
05:41:12 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialStartTime', 26102278.14, '127.0.0.1', 4022]
05:41:12 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ITIStartTimeHarp', 26102278.143488, '127.0.0.1', 4022]
05:41:12 PM:WARNING:warnings:warnings.py:_showwarnmsg:line 110:C:\Users\svc_aind_behavior\Documents\GitHub\dynamic-foraging-task\src\foraging_gui\Visualization.py:160: RuntimeWarning: Mean of empty slice
  RewardedHistoryF[i+1-self.kernel_size]=np.nanmean(RewardedHistoryT[i+1-self.kernel_size:i+1])

05:41:13 PM:INFO:Foraging:Foraging.py:_Save:line 2452:Saving current session, ForceSave=1
05:41:13 PM:INFO:Foraging:Foraging.py:_GetWaterCalibration:line 1046:Loaded Water Calibration
05:41:13 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 90:processing:C:\behavior_data\447-1-C\764788\behavior_764788_2024-11-07_16-26-33
05:41:13 PM:INFO:GenerateMetadata:GenerateMetadata.py:_handle_edge_cases:line 398:Cannot log reward delivery in session metadata - missing fields
05:41:13 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_reward_delivery:line 1235:No reward delivery metadata found!
05:41:13 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 908:No ephys data stream detected!
05:41:13 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 627:No photometry data stream detected!
05:41:13 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_high_speed_camera_stream:line 605:No camera data stream detected!
05:41:13 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 117:Session metadata generated successfully: True
05:41:13 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 118:Rig metadata generated successfully: True
05:41:13 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 119:Data description generated successfully: True
05:41:15 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/DelayStartTime', 26102280.848, '127.0.0.1', 4022]
05:41:15 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26102280.850496, '127.0.0.1', 4022]
05:41:16 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102281.407008, '127.0.0.1', 4023]
05:41:18 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/DelayStartTime', 26102284.114976, '127.0.0.1', 4022]
05:41:18 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26102284.11648, '127.0.0.1', 4022]
05:41:19 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTime', 26102285.097984, '127.0.0.1', 4022]
05:41:19 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTimeSoundCard', 26102285.101376, 4, '127.0.0.1']
05:41:19 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26102285.10048, '127.0.0.1', 4022]
05:41:20 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26102285.407808, '127.0.0.1', 4023]
05:41:20 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcomeTime', 26102285.406976, '127.0.0.1', 4022]
05:41:20 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcome', 'ErrorLeft', '127.0.0.1', 4022]
05:41:20 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26102285.619648, '127.0.0.1', 4023]
05:41:21 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102286.191968, '127.0.0.1', 4023]
05:41:21 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialEndTime', 26102286.397984, '127.0.0.1', 4022]
05:41:21 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26102286.40048, '127.0.0.1', 4022]
05:41:21 PM:INFO:Foraging:Foraging.py:_StartTrialLoop:line 4204:Current trial: 489
05:41:21 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialStartTime', 26102286.474976, '127.0.0.1', 4022]
05:41:21 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ITIStartTimeHarp', 26102286.478496, '127.0.0.1', 4022]
05:41:21 PM:WARNING:warnings:warnings.py:_showwarnmsg:line 110:C:\Users\svc_aind_behavior\Documents\GitHub\dynamic-foraging-task\src\foraging_gui\Visualization.py:160: RuntimeWarning: Mean of empty slice
  RewardedHistoryF[i+1-self.kernel_size]=np.nanmean(RewardedHistoryT[i+1-self.kernel_size:i+1])

05:41:21 PM:INFO:Foraging:Foraging.py:_Save:line 2452:Saving current session, ForceSave=1
05:41:21 PM:INFO:Foraging:Foraging.py:_GetWaterCalibration:line 1046:Loaded Water Calibration
05:41:21 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 90:processing:C:\behavior_data\447-1-C\764788\behavior_764788_2024-11-07_16-26-33
05:41:21 PM:INFO:GenerateMetadata:GenerateMetadata.py:_handle_edge_cases:line 398:Cannot log reward delivery in session metadata - missing fields
05:41:21 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_reward_delivery:line 1235:No reward delivery metadata found!
05:41:21 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 908:No ephys data stream detected!
05:41:21 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 627:No photometry data stream detected!
05:41:21 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_high_speed_camera_stream:line 605:No camera data stream detected!
05:41:21 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 117:Session metadata generated successfully: True
05:41:21 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 118:Rig metadata generated successfully: True
05:41:21 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 119:Data description generated successfully: True
05:41:23 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102288.854496, '127.0.0.1', 4023]
05:41:24 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/DelayStartTime', 26102290.132, '127.0.0.1', 4022]
05:41:24 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26102290.133504, '127.0.0.1', 4022]
05:41:25 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102290.732512, '127.0.0.1', 4023]
05:41:25 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102290.85184, '127.0.0.1', 4023]
05:41:29 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/DelayStartTime', 26102294.397984, '127.0.0.1', 4022]
05:41:29 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26102294.40048, '127.0.0.1', 4022]
05:41:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTime', 26102295.381984, '127.0.0.1', 4022]
05:41:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26102295.383488, '127.0.0.1', 4022]
05:41:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTimeSoundCard', 26102295.38432, 4, '127.0.0.1']
05:41:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26102295.751008, '127.0.0.1', 4023]
05:41:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcomeTime', 26102295.749984, '127.0.0.1', 4022]
05:41:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcome', 'ErrorLeft', '127.0.0.1', 4022]
05:41:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102295.980384, '127.0.0.1', 4023]
05:41:31 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialEndTime', 26102296.748, '127.0.0.1', 4022]
05:41:31 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26102296.750496, '127.0.0.1', 4022]
05:41:31 PM:INFO:Foraging:Foraging.py:_StartTrialLoop:line 4204:Current trial: 490
05:41:31 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialStartTime', 26102296.824, '127.0.0.1', 4022]
05:41:31 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ITIStartTimeHarp', 26102296.827488, '127.0.0.1', 4022]
05:41:31 PM:WARNING:warnings:warnings.py:_showwarnmsg:line 110:C:\Users\svc_aind_behavior\Documents\GitHub\dynamic-foraging-task\src\foraging_gui\Visualization.py:160: RuntimeWarning: Mean of empty slice
  RewardedHistoryF[i+1-self.kernel_size]=np.nanmean(RewardedHistoryT[i+1-self.kernel_size:i+1])

05:41:31 PM:INFO:Foraging:Foraging.py:_Save:line 2452:Saving current session, ForceSave=1
05:41:31 PM:INFO:Foraging:Foraging.py:_GetWaterCalibration:line 1046:Loaded Water Calibration
05:41:31 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 90:processing:C:\behavior_data\447-1-C\764788\behavior_764788_2024-11-07_16-26-33
05:41:31 PM:INFO:GenerateMetadata:GenerateMetadata.py:_handle_edge_cases:line 398:Cannot log reward delivery in session metadata - missing fields
05:41:31 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_reward_delivery:line 1235:No reward delivery metadata found!
05:41:31 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 908:No ephys data stream detected!
05:41:31 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 627:No photometry data stream detected!
05:41:31 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_high_speed_camera_stream:line 605:No camera data stream detected!
05:41:31 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 117:Session metadata generated successfully: True
05:41:31 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 118:Rig metadata generated successfully: True
05:41:31 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 119:Data description generated successfully: True
05:41:32 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/DelayStartTime', 26102297.932, '127.0.0.1', 4022]
05:41:32 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26102297.933504, '127.0.0.1', 4022]
05:41:33 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26102298.820512, '127.0.0.1', 4023]
05:41:34 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102299.542944, '127.0.0.1', 4023]
05:41:34 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/DelayStartTime', 26102299.932, '127.0.0.1', 4022]
05:41:34 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26102299.933504, '127.0.0.1', 4022]
05:41:35 PM:INFO:bias_indicator:bias_indicator.py:calculate_bias:line 139:Bias: -0.31229216277234473 Trial Count: 489
05:41:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTime', 26102300.914976, '127.0.0.1', 4022]
05:41:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26102300.91648, '127.0.0.1', 4022]
05:41:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTimeSoundCard', 26102300.91728, 4, '127.0.0.1']
05:41:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26102301.208704, '127.0.0.1', 4023]
05:41:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcomeTime', 26102301.208, '127.0.0.1', 4022]
05:41:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcome', 'ErrorLeft', '127.0.0.1', 4022]
05:41:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26102301.694304, '127.0.0.1', 4023]
05:41:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26102301.9032, '127.0.0.1', 4023]
05:41:37 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialEndTime', 26102302.197984, '127.0.0.1', 4022]
05:41:37 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 26102302.20048, '127.0.0.1', 4022]
05:41:37 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26102302.4608, '127.0.0.1', 4023]
05:41:41 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102306.914112, '127.0.0.1', 4023]
05:41:43 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102308.342752, '127.0.0.1', 4023]
05:41:53 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102318.332928, '127.0.0.1', 4023]
05:41:53 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102318.692096, '127.0.0.1', 4023]
05:41:57 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102322.218432, '127.0.0.1', 4023]
05:42:02 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102327.289568, '127.0.0.1', 4023]
05:42:02 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102327.760416, '127.0.0.1', 4023]
05:42:05 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102330.838432, '127.0.0.1', 4023]
05:42:06 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102331.247776, '127.0.0.1', 4023]
05:42:11 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102336.200896, '127.0.0.1', 4023]
05:42:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102360.446304, '127.0.0.1', 4023]
05:42:54 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102379.2104, '127.0.0.1', 4023]
05:42:59 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102384.250176, '127.0.0.1', 4023]
05:43:06 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102391.809152, '127.0.0.1', 4023]
05:43:38 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102424.080736, '127.0.0.1', 4023]
05:43:42 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102428.017248, '127.0.0.1', 4023]
05:43:43 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 26102428.791744, '127.0.0.1', 4023]
05:43:44 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102429.332352, '127.0.0.1', 4023]
05:43:44 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102429.552256, '127.0.0.1', 4023]
05:43:46 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102431.9128, '127.0.0.1', 4023]
05:43:49 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102434.472672, '127.0.0.1', 4023]
05:43:52 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102437.644512, '127.0.0.1', 4023]
05:44:20 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102465.298624, '127.0.0.1', 4023]
05:44:20 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102465.505184, '127.0.0.1', 4023]
05:44:20 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102465.68048, '127.0.0.1', 4023]
05:44:24 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102470.0464, '127.0.0.1', 4023]
05:45:25 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102530.514496, '127.0.0.1', 4023]
05:45:25 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102530.682144, '127.0.0.1', 4023]
05:45:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102535.404832, '127.0.0.1', 4023]
05:45:38 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102543.39184, '127.0.0.1', 4023]
05:45:54 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightLickTime', 26102559.560384, '127.0.0.1', 4023]
05:46:51 PM:ERROR:Foraging:Foraging.py:_StartTrialLoop:line 4326:trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1

alexpiet commented 1 week ago

This issue reoccurred:

file: Foraging.py
function: _StartTrialLoop
line 4284
logfile: W10DT714688/W10DT714688-D_gui_log_2024-11-15_09-25-27.txt
boxes: 2C, 8D
count: 2
text: trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1

GUI Log snippet (click for full log)

11:09:35 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288075.962112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:37 AM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 913:Percentage of left lick intervals under 100 ms is 5.05%.
11:09:37 AM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 919:Percentage of right lick intervals under 100 ms is 1.01%.
11:09:37 AM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 926:Percentage of right and left lick intervals under 100 ms is 2.33%.
11:09:37 AM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 951:Percentage of cross side lick intervals under 100 ms is 2.94%.
11:09:37 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288077.696768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:37 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288077.962112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:39 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288079.696736, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:39 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288079.962112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:41 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288081.746752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:41 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288082.012096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:43 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288083.746752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:43 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288084.012096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:45 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288085.796736, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:45 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288086.062112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:47 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288087.846752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:47 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288088.112096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:49 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288089.846752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:49 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288090.112096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:51 AM:ERROR:Foraging:Foraging.py:_StartTrialLoop:line 4284:trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1

 Full GUI log (first 200 and last 200 lines)

09:25:27 AM:INFO:Foraging:Foraging.py:start_gui_log_file:line 4679:Starting logfile!
09:25:27 AM:INFO:Foraging:Foraging.py:log_git_hash:line 4690:Python version: 3.11.9 | packaged by Anaconda, Inc. | (main, Apr 19 2024, 16:40:41) [MSC v.1916 64 bit (AMD64)]
09:25:27 AM:INFO:Foraging:Foraging.py:log_git_hash:line 4707:Current git commit branch, hash: main, 60f8beb5cf259b096e71331217b2a631d1eaeb43
09:25:27 AM:INFO:Foraging:Foraging.py:log_git_hash:line 4711:Current foraging_gui version: 1.6.9
09:25:27 AM:WARNING:Foraging:Foraging.py:log_git_hash:line 4721:local repository is clean
09:25:27 AM:INFO:Foraging:Foraging.py:<module>:line 4822:Setting QApplication attributes
09:25:27 AM:INFO:Foraging:Foraging.py:<module>:line 4830:Starting QApplication and Window
09:25:27 AM:INFO:Foraging:Foraging.py:__init__:line 69:Creating Window
09:25:27 AM:INFO:Foraging:Foraging.py:_GetSettings:line 1166:Loaded settings_box file
09:25:27 AM:INFO:Foraging:Foraging.py:_GetSettings:line 1194:Loaded settings file
09:25:27 AM:WARNING:Foraging:Foraging.py:_GetSettings:line 1204:Missing setting (FIP_settings), using default: C:\Users\svc_aind_behavior\Documents\FIPSettings
09:25:27 AM:WARNING:Foraging:Foraging.py:_GetSettings:line 1204:Missing setting (bonsai_config_path), using default: C:\Users\svc_aind_behavior\Documents\GitHub\dynamic-foraging-task\bonsai\Bonsai.config
09:25:27 AM:WARNING:Foraging:Foraging.py:_GetSettings:line 1204:Missing setting (metadata_dialog_folder), using default: C:\Users\svc_aind_behavior\Documents\ForagingSettings\metadata_dialog\
09:25:27 AM:WARNING:Foraging:Foraging.py:_GetSettings:line 1204:Missing setting (rig_metadata_folder), using default: C:\Users\svc_aind_behavior\Documents\ForagingSettings\rig_metadata\
09:25:27 AM:WARNING:Foraging:Foraging.py:_GetSettings:line 1204:Missing setting (project_info_file), using default: C:\Users\svc_aind_behavior\Documents\ForagingSettings\Project Name and Funding Source v2.csv
09:25:27 AM:WARNING:Foraging:Foraging.py:_GetSettings:line 1204:Missing setting (schedule_path), using default: Z:\dynamic_foraging\DynamicForagingSchedule.csv
09:25:27 AM:WARNING:Foraging:Foraging.py:_GetSettings:line 1204:Missing setting (go_cue_decibel_box1), using default: 60
09:25:27 AM:WARNING:Foraging:Foraging.py:_GetSettings:line 1204:Missing setting (go_cue_decibel_box2), using default: 60
09:25:27 AM:WARNING:Foraging:Foraging.py:_GetSettings:line 1204:Missing setting (go_cue_decibel_box3), using default: 60
09:25:27 AM:WARNING:Foraging:Foraging.py:_GetSettings:line 1204:Missing setting (lick_spout_distance_box1), using default: 5000
09:25:27 AM:WARNING:Foraging:Foraging.py:_GetSettings:line 1204:Missing setting (lick_spout_distance_box2), using default: 5000
09:25:27 AM:WARNING:Foraging:Foraging.py:_GetSettings:line 1204:Missing setting (lick_spout_distance_box3), using default: 5000
09:25:27 AM:WARNING:Foraging:Foraging.py:_GetSettings:line 1204:Missing setting (lick_spout_distance_box4), using default: 5000
09:25:27 AM:WARNING:Foraging:Foraging.py:_GetSettings:line 1204:Missing setting (name_mapper_file), using default: C:\Users\svc_aind_behavior\Documents\ForagingSettings\name_mapper.json
09:25:27 AM:WARNING:Foraging:Foraging.py:_GetSettings:line 1204:Missing setting (create_rig_metadata), using default: True
09:25:27 AM:WARNING:Foraging:Foraging.py:_GetSettings:line 1204:Missing setting (save_each_trial), using default: True
09:25:27 AM:WARNING:Foraging:Foraging.py:_GetSettings:line 1204:Missing setting (AutomaticUpload), using default: True
09:25:27 AM:WARNING:Foraging:Foraging.py:_GetSettings:line 1204:Missing setting (manifest_flag_dir), using default: C:\Users\svc_aind_behavior\Documents\aind_watchdog_service\manifest
09:25:27 AM:WARNING:Foraging:Foraging.py:_GetSettings:line 1204:Missing setting (auto_engage), using default: True
09:25:27 AM:WARNING:Foraging:Foraging.py:_GetSettings:line 1204:Missing setting (clear_figure_after_save), using default: True
09:25:27 AM:WARNING:Foraging:Foraging.py:_GetSettings:line 1204:Missing setting (add_default_project_name), using default: True
09:25:27 AM:INFO:Foraging:Foraging.py:_LoadSchedule:line 1092:Loaded behavior schedule
09:25:27 AM:INFO:Foraging:Foraging.py:_GetWaterCalibration:line 1056:Loaded Water Calibration
09:25:27 AM:INFO:Foraging:Foraging.py:_load_most_recent_rig_json:line 1526:Found existing rig.json: rig_446-8-D_2024-11-12_11_02_49.json
09:25:27 AM:INFO:RigJsonBuilder:RigJsonBuilder.py:build_rig_json_core:line 60:building rig json
09:25:27 AM:INFO:RigJsonBuilder:RigJsonBuilder.py:build_rig_json_core:line 624:finished building rig json
09:25:27 AM:INFO:RigJsonBuilder:RigJsonBuilder.py:build_rig_json:line 53:Using existing rig json
09:25:27 AM:INFO:Foraging:Foraging.py:_LoadUI:line 275:Using ForagingGUI.ui interface
09:25:27 AM:INFO:Foraging:Foraging.py:__init__:line 119:Setting Window title: 446-8-D
09:25:27 AM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1363:Trying to connect to already running Bonsai
09:25:27 AM:INFO:Foraging:Foraging.py:_ConnectOSC:line 1410:connecting to GUI and Bonsai through OSC
09:25:28 AM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1367:Could not connect: timed out
09:25:28 AM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1376:Starting Bonsai
09:25:28 AM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4794:BONSAI logging starting
09:25:29 AM:INFO:Foraging:Foraging.py:_ConnectOSC:line 1410:connecting to GUI and Bonsai through OSC
09:25:30 AM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1390:Could not connect, total waiting time 0.5 seconds: timed out
09:25:30 AM:INFO:Foraging:Foraging.py:_ConnectOSC:line 1410:connecting to GUI and Bonsai through OSC
09:25:31 AM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1390:Could not connect, total waiting time 1.0 seconds: timed out
09:25:32 AM:INFO:Foraging:Foraging.py:_ConnectOSC:line 1410:connecting to GUI and Bonsai through OSC
09:25:32 AM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1393:Connected to Bonsai after 1.5 seconds
09:25:32 AM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1394:Bonsai started successfully
09:25:32 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11281832.717536, '127.0.0.1', 4033], displaying at 0.5 Hz
09:25:32 AM:INFO:Foraging:Foraging.py:_ConnectSlims:line 1273:Attempting to connect to Slims
09:25:32 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 11281832.725504, '127.0.0.1', 4032]
09:25:33 AM:INFO:Foraging:Foraging.py:_ConnectSlims:line 1290:Successfully connected to Slims
09:25:33 AM:WARNING:Dialogs:Dialogs.py:_LoadCalibrationParameters:line 507:could not find water calibration parameters: C:\Users\svc_aind_behavior\Documents\ForagingSettings\WaterCalibrationPar_4.json
09:25:33 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1881:processing parameter changes
09:25:33 AM:INFO:Foraging:Foraging.py:_InitializeMotorStage:line 867:Scanning for newscale stages
09:25:33 AM:INFO:Foraging:Foraging.py:_InitializeMotorStage:line 877:Could not find any instances of NewScale Stage
09:25:33 AM:INFO:Foraging:Foraging.py:_insert_stage_widget:line 255:Inserting Stage Widget
09:25:33 AM:INFO:connection:connection.py:_connect:line 688:Connecting to aibspi(10.128.108.106):2181, use_ssl: False
09:25:33 AM:INFO:client:client.py:_session_callback:line 561:Zookeeper connection established, state: CONNECTED
09:25:33 AM:INFO:connection:connection.py:_connect_attempt:line 655:Closing connection to aibspi:2181
09:25:33 AM:INFO:client:client.py:_session_callback:line 567:Zookeeper session closed, state: CLOSED
09:25:33 AM:INFO:stepper_motor_device:stepper_motor_device.py:__init__:line 68:Connected to COM4, device id: 1130
09:25:33 AM:INFO:stepper_motor_device:stepper_motor_device.py:load_registers:line 149:Getting available registers on device
09:25:33 AM:INFO:stage_model:stage_model.py:configure_device:line 81:Configuring registers with default values (can be modified in config file)
09:25:33 AM:INFO:move_worker:move_worker.py:goto_origin:line 194:(move to origin): Moving stage to origin
09:25:33 AM:INFO:move_worker:move_worker.py:move_relative:line 336:(move relative): Moved motors by steps specified in [0, -27200, -27200, 0]
09:25:33 AM:INFO:move_worker:move_worker.py:move_relative:line 336:(move relative): Moved motors by steps specified in [-27200, 0, 0, 0]
09:25:33 AM:INFO:move_worker:move_worker.py:move_relative:line 336:(move relative): Moved motors by steps specified in [0, 0, 0, -27200]
09:25:40 AM:INFO:stage_model:stage_model.py:configure_device:line 87:Zeroed all motor positions
09:25:40 AM:INFO:stage_model:stage_model.py:configure_device:line 88:Finished configuring stepper motor device
#############################################################
09:25:40 AM:INFO:Foraging:Foraging.py:_StopPhotometry:line 3446:Checking that photometry is not running
09:25:40 AM:INFO:Foraging:Foraging.py:_StopPhotometry:line 3456:Photometry excitation stopped
09:25:40 AM:INFO:Foraging:Foraging.py:_load_most_recent_rig_json:line 1526:Found existing rig.json: rig_446-8-D_2024-11-12_11_02_49.json
09:25:40 AM:INFO:Foraging:Foraging.py:_ReconnectBonsai:line 963:attempting to close bonsai connection
09:25:43 AM:INFO:Foraging:Foraging.py:_ReconnectBonsai:line 971:bonsai connection closed
09:25:43 AM:INFO:Foraging:Foraging.py:_ReconnectBonsai:line 973:attempting to restart bonsai
09:25:43 AM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1363:Trying to connect to already running Bonsai
09:25:43 AM:INFO:Foraging:Foraging.py:_ConnectOSC:line 1410:connecting to GUI and Bonsai through OSC
09:25:43 AM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1370:Connected to already running Bonsai
09:25:43 AM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1371:Bonsai started successfully
09:25:43 AM:INFO:Foraging:Foraging.py:__init__:line 219:Start up complete
09:25:44 AM:INFO:aws_util:aws_util.py:import_df_from_s3:line 115:Dataframe imported from s3://aind-behavior-data/foraging_nwb_bonsai_processed/df_sessions.pkl, len(df) = 5403
09:25:44 AM:INFO:aws_util:aws_util.py:import_df_from_s3:line 115:Dataframe imported from s3://aind-behavior-data/foraging_auto_training/df_manager_447_demo.pkl, len(df) = 3963
09:25:45 AM:INFO:aws_util:aws_util.py:download_dir_from_s3:line 140:134 objects downloaded from s3://aind-behavior-data/foraging_auto_training/saved_curriculums/ to C:\Users\svc_aind_behavior/capsule/scratch/saved_curriculums/
09:25:45 AM:INFO:curriculum_manager:curriculum_manager.py:download_curriculums:line 162:Found 26 curriculums in C:\Users\svc_aind_behavior/capsule/scratch/saved_curriculums/
09:25:46 AM:INFO:aws_util:aws_util.py:download_dir_from_s3:line 140:134 objects downloaded from s3://aind-behavior-data/foraging_auto_training/saved_curriculums/ to C:\Users\svc_aind_behavior/.aind_auto_train/curriculum_manager/
09:25:46 AM:INFO:curriculum_manager:curriculum_manager.py:download_curriculums:line 162:Found 26 curriculums in C:\Users\svc_aind_behavior/.aind_auto_train/curriculum_manager/
09:25:46 AM:INFO:Dialogs:Dialogs.py:update_auto_train_fields:line 2386:No entry found in df_training_manager for subject_id: 0
09:25:46 AM:INFO:Dialogs:Dialogs.py:_override_curriculum_clicked:line 2734:Override stage clicked: state=2
09:25:46 AM:INFO:Dialogs:Dialogs.py:_update_stage_to_apply:line 2760:Current stage to apply: unknown training stage @unknown curriculum
09:27:08 AM:INFO:Dialogs:Dialogs.py:_start_preview:line 1326:Camera is on
09:27:12 AM:INFO:Foraging:Foraging.py:_StopCurrentSession:line 3599:Stopping current trials
09:27:27 AM:INFO:Foraging:Foraging.py:_Open:line 2966:Quick load success: C:\behavior_data\446-8-D\746345\behavior_746345_2024-11-14_08-40-23\behavior\746345_2024-11-14_08-40-23.json
09:27:27 AM:INFO:Foraging:Foraging.py:_NewSession:line 3509:New Session pressed
09:27:27 AM:INFO:Foraging:Foraging.py:_NewSession:line 3541:Resetting session run flag
09:27:27 AM:INFO:Foraging:Foraging.py:_StopPhotometry:line 3446:Checking that photometry is not running
09:27:27 AM:INFO:Foraging:Foraging.py:_StopPhotometry:line 3456:Photometry excitation stopped
09:27:27 AM:INFO:Foraging:Foraging.py:_NewSession:line 3579:New Session complete
09:27:28 AM:INFO:Foraging:Foraging.py:_QComboBoxUpdate:line 1850:Field updated: Photometry:on
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1881:processing parameter changes
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: RightValue, 0.03 -> 0.033
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: LeftValue, 0.03 -> 0.034
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: ID, 0.0 -> 746345.0
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: RightValue, 0.03 -> 0.033
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: LeftValue, 0.03 -> 0.034
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: GiveWaterL, 0.03 -> 0.034
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: GiveWaterR, 0.03 -> 0.033
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1881:processing parameter changes
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: DelayMin, 0.0 -> 1.0
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: DelayBeta, 1.0 -> 0.0
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: DelayMin, 0.0 -> 1.0
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: DelayBeta, 1.0 -> 0.0
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1881:processing parameter changes
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1881:processing parameter changes
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: BlockMax, 60.0 -> 35.0
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: RewardDelay, 0.0 -> 0.1
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: ITIMax, 8.0 -> 10.0
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: ITIBeta, 2.0 -> 3.0
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: ResponseTime, 1.0 -> 1.5
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: BlockMax, 60.0 -> 35.0
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: RewardDelay, 0.0 -> 0.1
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: ITIMax, 8.0 -> 10.0
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: ITIBeta, 2.0 -> 3.0
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: ResponseTime, 1.0 -> 1.5
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1881:processing parameter changes
09:27:28 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1881:processing parameter changes
09:27:28 AM:WARNING:warnings:warnings.py:_showwarnmsg:line 110:C:\Users\svc_aind_behavior\Documents\GitHub\dynamic-foraging-task\src\foraging_gui\Visualization.py:160: RuntimeWarning: Mean of empty slice
  RewardedHistoryF[i+1-self.kernel_size]=np.nanmean(RewardedHistoryT[i+1-self.kernel_size:i+1])

09:27:34 AM:INFO:Foraging:Foraging.py:_check_drop_frames:line 674:
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1881:processing parameter changes
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: Ignored, 100.0 -> 10.0
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: BlockBeta, 20.0 -> 10.0
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: StopIgnores, 20.0 -> 25.0
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: MaxTime, 120.0 -> 75.0
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: RewardConsumeTime, 3.0 -> 1.0
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: Multiplier, 0.8 -> 0.5
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: RightValue_volume, 3.0 -> 2.0
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: RightValue, 0.033 -> 0.022
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: LeftValue, 0.034 -> 0.023
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: Unrewarded, 200.0 -> 10.0
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: LeftValue_volume, 3.0 -> 2.0
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: warm_min_trial, 60.0 -> 50.0
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: Ignored, 100.0 -> 10.0
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: BlockBeta, 20.0 -> 10.0
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: StopIgnores, 20.0 -> 25.0
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: MaxTime, 120.0 -> 75.0
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: RewardConsumeTime, 3.0 -> 1.0
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: Multiplier, 0.8 -> 0.5
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: RightValue_volume, 3.0 -> 2.0
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: RightValue, 0.033 -> 0.022
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: LeftValue, 0.034 -> 0.023
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: Unrewarded, 200.0 -> 10.0
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: LeftValue_volume, 3.0 -> 2.0
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: warm_min_trial, 60.0 -> 50.0
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: lineEdit_y1, 0.0 -> 20.5
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: lineEdit_z, 0.0 -> 19.5
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: lineEdit_y2, 0.0 -> 20.5
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: lineEdit_x, 0.0 -> 16.1975
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: lineEdit_step_size, 1.0 -> 0.2
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: BaseWeight,  -> 30.0
09:27:34 AM:INFO:Foraging:Foraging.py:keyPressEvent:line 1940:Changing parameter: TargetWeight,  -> 25.5
09:27:34 AM:INFO:Dialogs:Dialogs.py:update_auto_train_lock:line 2938:AutoTrain disengaged!
09:27:34 AM:INFO:curriculum_manager:curriculum_manager.py:get_curriculum:line 131:Loaded a DynamicForagingCurriculum model from 'Uncoupled Without Baiting_curriculum_v2.3.1rwdDelay159_schema_v1.0.json'.
09:27:34 AM:INFO:Dialogs:Dialogs.py:_override_curriculum_clicked:line 2734:Override stage clicked: state=0
09:27:34 AM:INFO:Dialogs:Dialogs.py:_update_curriculum_diagrams:line 2680:Selected curriculum: {'curriculum_name': 'Uncoupled Without Baiting', 'curriculum_version': '2.3.1rwdDelay159', 'curriculum_schema_version': '1.0', 'curriculum_description': '2024-08-16 max_len = 75 mins; decrease finished trial criterion'}
09:27:34 AM:INFO:curriculum_manager:curriculum_manager.py:get_curriculum:line 131:Loaded a DynamicForagingCurriculum model from 'Uncoupled Without Baiting_curriculum_v2.3.1rwdDelay159_schema_v1.0.json'.
09:27:34 AM:INFO:Dialogs:Dialogs.py:_update_stage_to_apply:line 2760:Current stage to apply: STAGE_3 @Uncoupled Without Baiting (v2.3.1rwdDelay159@1.0)
09:27:34 AM:INFO:Dialogs:Dialogs.py:_update_stage_to_apply:line 2760:Current stage to apply: STAGE_3 @Uncoupled Without Baiting (v2.3.1rwdDelay159@1.0)
09:27:34 AM:INFO:Dialogs:Dialogs.py:_update_curriculum_diagrams:line 2680:Selected curriculum: {'curriculum_name': 'Uncoupled Without Baiting', 'curriculum_version': '2.3.1rwdDelay159', 'curriculum_schema_version': '1.0', 'curriculum_description': '2024-08-16 max_len = 75 mins; decrease finished trial criterion'}
09:27:34 AM:INFO:curriculum_manager:curriculum_manager.py:get_curriculum:line 131:Loaded a DynamicForagingCurriculum model from 'Uncoupled Without Baiting_curriculum_v2.3.1rwdDelay159_schema_v1.0.json'.
09:27:34 AM:INFO:Dialogs:Dialogs.py:update_auto_train_lock:line 2887:AutoTrain engaged! STAGE_3 @ Uncoupled Without Baiting (v2.3.1rwdDelay159@1.0)
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3018: Widget "training_stage" not found. skipped...
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3010:Task is set to Coupled Without Baiting
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3018: Widget "task_url" not found. skipped...
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3018: Widget "task_schema_version" not found. skipped...
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3018: Widget "description" not found. skipped...
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3065:BaseRewardSum is set to 0.8
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3065:RewardFamily is set to 1
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3065:RewardPairsN is set to 1
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3030: Widget "UncoupledReward" has been disabled by the GUI. skipped...
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3065:Randomness is set to Exponential
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3065:BlockMin is set to 20
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3065:BlockMax is set to 35
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3065:BlockBeta is set to 10
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3065:BlockMinReward is set to 0
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3065:DelayMin is set to 1.0
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3065:DelayMax is set to 1.0
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3065:DelayBeta is set to 0.0
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3065:RewardDelay is set to 0.1
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3065:AutoReward is set to True
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3065:AutoWaterType is set to Natural
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3065:Multiplier is set to 0.5
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3065:Unrewarded is set to 10
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3065:Ignored is set to 10
09:27:34 AM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 3065:ITIMin is set to 1.0
_____SKIPPING_____
11:06:32 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287893.162144, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:34 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287894.8968, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:34 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287895.212128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:36 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287896.946784, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:36 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287897.262144, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:38 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287898.996768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:38 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287899.312128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:40 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287901.046784, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:40 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287901.312128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:42 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287903.096768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:43 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287903.362112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:44 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287905.146784, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:45 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287905.412128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:46 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287907.196768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:47 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287907.462112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:48 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287909.246752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:49 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287909.512128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:50 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287911.2968, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:51 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287911.562144, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:53 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287913.346784, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:53 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287913.612128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:55 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287915.396768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:55 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287915.662112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:57 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287917.3968, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:57 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287917.712128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:59 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287919.446784, '127.0.0.1', 4033], displaying at 0.5 Hz
11:06:59 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287919.762144, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:01 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287921.496768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:01 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287921.812128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:03 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287923.546784, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:03 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287923.862144, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:05 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287925.546784, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:05 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287925.862144, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:07 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287927.596768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:07 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287927.912128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:09 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287929.5968, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:09 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287929.912128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:11 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287931.646784, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:11 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287931.962144, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:13 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287933.646752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:13 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287933.962112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:15 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287935.6968, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:15 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287936.012096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:17 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287937.746784, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:17 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287938.062144, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:19 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287939.746752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:19 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287940.062112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:21 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287941.7968, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:21 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287942.112096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:23 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287943.796768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:23 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287944.112128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:25 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287945.846752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:25 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287946.162112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:27 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287947.8968, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:27 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287948.212096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:29 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287949.896768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:29 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287950.212128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:31 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287951.946752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:31 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287952.212128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:33 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287953.9968, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:33 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287954.212096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:35 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287956.046784, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:35 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287956.262144, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:37 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287958.096768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:37 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287958.312128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:39 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287960.096768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:39 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287960.312096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:41 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287962.146784, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:42 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287962.362144, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:43 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287964.146752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:44 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287964.362112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:45 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287966.196768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:46 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287966.412096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:47 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287968.246784, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:48 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287968.462144, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:49 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287970.246752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:50 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287970.512128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:51 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287972.296768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:52 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287972.562112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:53 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287974.296768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:54 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287974.562144, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:56 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287976.346752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:56 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287976.612128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:58 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287978.396768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:07:58 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287978.612096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:00 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287980.396768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:00 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287980.662144, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:02 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287982.396768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:02 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287982.662112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:04 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287984.446784, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:04 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287984.712128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:06 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287986.496768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:06 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287986.762144, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:08 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287988.496768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:08 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287988.762112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:10 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287990.546752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:10 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287990.812096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:12 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287992.596768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:12 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287992.862112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:14 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287994.596768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:14 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287994.862112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:16 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287996.646752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:16 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287996.912096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:18 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11287998.646784, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:18 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11287998.912128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:20 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288000.696768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:20 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288000.962112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:22 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288002.746752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:22 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288003.012096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:24 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288004.746784, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:24 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288005.012096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:26 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288006.796768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:26 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288007.062112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:28 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288008.796736, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:28 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288009.062112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:30 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288010.846784, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:30 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288011.112096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:32 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288012.846752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:32 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288013.162112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:34 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288014.846752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:34 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288015.162112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:36 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288016.896768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:36 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288017.212096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:38 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288018.896768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:38 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288019.212128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:40 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288020.946752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:40 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288021.262112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:42 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288022.996768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:42 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288023.312096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:44 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288024.996768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:44 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288025.312128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:46 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288027.046752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:47 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288027.362112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:48 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288029.046752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:49 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288029.36208, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:50 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288031.096768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:51 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288031.412128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:52 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288033.146752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:53 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288033.462112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:54 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288035.146784, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:55 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288035.462112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:56 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288037.196768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:57 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288037.512128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:58 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288039.196736, '127.0.0.1', 4033], displaying at 0.5 Hz
11:08:59 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288039.512096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:00 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288041.196736, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:01 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288041.512096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:02 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288043.196768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:03 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288043.512128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:04 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288045.246752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:05 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288045.562112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:06 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288047.296736, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:07 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288047.612096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:08 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288049.296768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:09 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288049.612128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:11 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288051.346752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:11 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288051.612096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:13 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288053.396736, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:13 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288053.66208, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:15 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288055.396768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:15 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288055.712128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:17 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288057.446752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:17 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288057.762112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:19 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288059.446752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:19 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288059.76208, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:21 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288061.496768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:21 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288061.812128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:23 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288063.496736, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:23 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288063.812096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:25 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288065.546752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:25 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288065.86208, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:27 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288067.596768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:27 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288067.862112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:29 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288069.596736, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:29 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288069.862112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:31 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288071.646752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:31 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288071.912128, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:33 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288073.646752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:33 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288073.962112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:35 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288075.696736, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:35 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288075.962112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:37 AM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 913:Percentage of left lick intervals under 100 ms is 5.05%.
11:09:37 AM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 919:Percentage of right lick intervals under 100 ms is 1.01%.
11:09:37 AM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 926:Percentage of right and left lick intervals under 100 ms is 2.33%.
11:09:37 AM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 951:Percentage of cross side lick intervals under 100 ms is 2.94%.
11:09:37 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288077.696768, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:37 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288077.962112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:39 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288079.696736, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:39 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288079.962112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:41 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288081.746752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:41 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288082.012096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:43 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288083.746752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:43 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288084.012096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:45 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288085.796736, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:45 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288086.062112, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:47 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288087.846752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:47 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288088.112096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:49 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryRising', 11288089.846752, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:49 AM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 11288090.112096, '127.0.0.1', 4033], displaying at 0.5 Hz
11:09:51 AM:ERROR:Foraging:Foraging.py:_StartTrialLoop:line 4284:trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1

alexpiet commented 1 day ago

This issue reoccurred:

file: Foraging.py
function: _StartTrialLoop
line 4266
logfile: W10DT714688/W10DT714688-D_gui_log_2024-11-23_17-44-58.txt
boxes: 3B, 3C, 6A, 7A, 8D
count: 5
text: trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1

GUI Log snippet (click for full log)

09:25:11 PM:WARNING:MyFunctions:MyFunctions.py:_CheckStop:line 1191:Stop because running time exceeds or equals: 75m
09:30:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 913:Percentage of left lick intervals under 100 ms is 0.79%.
09:30:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 919:Percentage of right lick intervals under 100 ms is 0.46%.
09:30:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 926:Percentage of right and left lick intervals under 100 ms is 0.68%.
09:30:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 951:Percentage of cross side lick intervals under 100 ms is 1.85%.
09:40:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 913:Percentage of left lick intervals under 100 ms is 0.79%.
09:40:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 919:Percentage of right lick intervals under 100 ms is 0.46%.
09:40:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 926:Percentage of right and left lick intervals under 100 ms is 0.68%.
09:40:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 951:Percentage of cross side lick intervals under 100 ms is 1.85%.
09:50:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 913:Percentage of left lick intervals under 100 ms is 0.79%.
09:50:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 919:Percentage of right lick intervals under 100 ms is 0.46%.
09:50:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 926:Percentage of right and left lick intervals under 100 ms is 0.68%.
09:50:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 951:Percentage of cross side lick intervals under 100 ms is 1.85%.
09:53:52 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 12017932.181824, '127.0.0.1', 4033]
09:53:52 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 12017932.282176, '127.0.0.1', 4033]
09:53:58 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 12017938.254944, '127.0.0.1', 4033]
09:53:58 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 12017938.427584, '127.0.0.1', 4033]
09:55:04 PM:INFO:Foraging:Foraging.py:_Start:line 3912:Start button pressed: ending trial loop
09:55:04 PM:INFO:Foraging:Foraging.py:_StartTrialLoop:line 4122:ending trial loop
09:55:07 PM:ERROR:Foraging:Foraging.py:_StartTrialLoop:line 4266:trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1

 Full GUI log (first 200 and last 200 lines)

05:44:58 PM:INFO:Foraging:Foraging.py:start_gui_log_file:line 4661:Starting logfile!
05:44:58 PM:INFO:Foraging:Foraging.py:log_git_hash:line 4672:Python version: 3.11.9 | packaged by Anaconda, Inc. | (main, Apr 19 2024, 16:40:41) [MSC v.1916 64 bit (AMD64)]
05:44:58 PM:INFO:Foraging:Foraging.py:log_git_hash:line 4689:Current git commit branch, hash: main, 1a7c9d3d1aa10284caed44ba9b223c6bfbb8ff5f
05:44:58 PM:INFO:Foraging:Foraging.py:log_git_hash:line 4693:Current foraging_gui version: 1.6.10
05:44:58 PM:WARNING:Foraging:Foraging.py:log_git_hash:line 4703:local repository is clean
05:44:58 PM:INFO:Foraging:Foraging.py:<module>:line 4804:Setting QApplication attributes
05:44:58 PM:INFO:Foraging:Foraging.py:<module>:line 4812:Starting QApplication and Window
05:44:58 PM:INFO:Foraging:Foraging.py:__init__:line 69:Creating Window
05:44:58 PM:INFO:Foraging:Foraging.py:_GetSettings:line 1174:Loaded settings_box file
05:44:58 PM:INFO:Foraging:Foraging.py:_GetSettings:line 1202:Loaded settings file
05:44:58 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1212:Missing setting (FIP_settings), using default: C:\Users\svc_aind_behavior\Documents\FIPSettings
05:44:58 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1212:Missing setting (bonsai_config_path), using default: C:\Users\svc_aind_behavior\Documents\GitHub\dynamic-foraging-task\bonsai\Bonsai.config
05:44:58 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1212:Missing setting (metadata_dialog_folder), using default: C:\Users\svc_aind_behavior\Documents\ForagingSettings\metadata_dialog\
05:44:58 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1212:Missing setting (rig_metadata_folder), using default: C:\Users\svc_aind_behavior\Documents\ForagingSettings\rig_metadata\
05:44:58 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1212:Missing setting (project_info_file), using default: C:\Users\svc_aind_behavior\Documents\ForagingSettings\Project Name and Funding Source v2.csv
05:44:58 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1212:Missing setting (schedule_path), using default: Z:\dynamic_foraging\DynamicForagingSchedule.csv
05:44:58 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1212:Missing setting (go_cue_decibel_box1), using default: 60
05:44:58 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1212:Missing setting (go_cue_decibel_box2), using default: 60
05:44:58 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1212:Missing setting (go_cue_decibel_box3), using default: 60
05:44:58 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1212:Missing setting (lick_spout_distance_box1), using default: 5000
05:44:58 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1212:Missing setting (lick_spout_distance_box2), using default: 5000
05:44:58 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1212:Missing setting (lick_spout_distance_box3), using default: 5000
05:44:58 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1212:Missing setting (lick_spout_distance_box4), using default: 5000
05:44:58 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1212:Missing setting (name_mapper_file), using default: C:\Users\svc_aind_behavior\Documents\ForagingSettings\name_mapper.json
05:44:58 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1212:Missing setting (create_rig_metadata), using default: True
05:44:58 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1212:Missing setting (save_each_trial), using default: True
05:44:58 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1212:Missing setting (AutomaticUpload), using default: True
05:44:58 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1212:Missing setting (manifest_flag_dir), using default: C:\Users\svc_aind_behavior\Documents\aind_watchdog_service\manifest
05:44:58 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1212:Missing setting (auto_engage), using default: True
05:44:58 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1212:Missing setting (clear_figure_after_save), using default: True
05:44:58 PM:WARNING:Foraging:Foraging.py:_GetSettings:line 1212:Missing setting (add_default_project_name), using default: True
05:44:58 PM:INFO:Foraging:Foraging.py:_LoadSchedule:line 1100:Loaded behavior schedule
05:44:58 PM:INFO:Foraging:Foraging.py:_GetWaterCalibration:line 1064:Loaded Water Calibration
05:44:58 PM:INFO:Foraging:Foraging.py:_load_most_recent_rig_json:line 1534:Found existing rig.json: rig_446-8-D_2024-11-19_11_21_56.json
05:44:58 PM:INFO:RigJsonBuilder:RigJsonBuilder.py:build_rig_json_core:line 60:building rig json
05:44:58 PM:INFO:RigJsonBuilder:RigJsonBuilder.py:build_rig_json_core:line 624:finished building rig json
05:44:58 PM:INFO:RigJsonBuilder:RigJsonBuilder.py:build_rig_json:line 53:Using existing rig json
05:44:58 PM:INFO:Foraging:Foraging.py:_LoadUI:line 275:Using ForagingGUI.ui interface
05:44:58 PM:INFO:Foraging:Foraging.py:__init__:line 119:Setting Window title: 446-8-D
05:44:58 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1371:Trying to connect to already running Bonsai
05:44:58 PM:INFO:Foraging:Foraging.py:_ConnectOSC:line 1418:connecting to GUI and Bonsai through OSC
05:44:59 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1375:Could not connect: timed out
05:44:59 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1384:Starting Bonsai
05:44:59 PM:INFO:Foraging:Foraging.py:log_subprocess_output:line 4776:BONSAI logging starting
05:45:00 PM:INFO:Foraging:Foraging.py:_ConnectOSC:line 1418:connecting to GUI and Bonsai through OSC
05:45:01 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1398:Could not connect, total waiting time 0.5 seconds: timed out
05:45:01 PM:INFO:Foraging:Foraging.py:_ConnectOSC:line 1418:connecting to GUI and Bonsai through OSC
05:45:02 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1398:Could not connect, total waiting time 1.0 seconds: timed out
05:45:03 PM:INFO:Foraging:Foraging.py:_ConnectOSC:line 1418:connecting to GUI and Bonsai through OSC
05:45:03 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1401:Connected to Bonsai after 1.5 seconds
05:45:03 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1402:Bonsai started successfully
05:45:03 PM:INFO:Foraging:Foraging.py:_ConnectSlims:line 1281:Attempting to connect to Slims
05:45:03 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 52:['/PhotometryFalling', 12003003.137536, '127.0.0.1', 4033], displaying at 0.5 Hz
05:45:03 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 12003003.145504, '127.0.0.1', 4032]
05:45:03 PM:INFO:Foraging:Foraging.py:_ConnectSlims:line 1298:Successfully connected to Slims
05:45:03 PM:WARNING:Dialogs:Dialogs.py:_LoadCalibrationParameters:line 480:could not find water calibration parameters: C:\Users\svc_aind_behavior\Documents\ForagingSettings\WaterCalibrationPar_4.json
05:45:04 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1863:processing parameter changes
05:45:04 PM:INFO:Foraging:Foraging.py:_InitializeMotorStage:line 875:Scanning for newscale stages
05:45:04 PM:INFO:Foraging:Foraging.py:_InitializeMotorStage:line 885:Could not find any instances of NewScale Stage
05:45:04 PM:INFO:Foraging:Foraging.py:_insert_stage_widget:line 255:Inserting Stage Widget
05:45:04 PM:INFO:connection:connection.py:_connect:line 688:Connecting to aibspi(10.128.108.106):2181, use_ssl: False
05:45:04 PM:INFO:client:client.py:_session_callback:line 561:Zookeeper connection established, state: CONNECTED
05:45:04 PM:INFO:connection:connection.py:_connect_attempt:line 655:Closing connection to aibspi:2181
05:45:04 PM:INFO:client:client.py:_session_callback:line 567:Zookeeper session closed, state: CLOSED
05:45:04 PM:INFO:stepper_motor_device:stepper_motor_device.py:__init__:line 68:Connected to COM4, device id: 1130
05:45:04 PM:INFO:stepper_motor_device:stepper_motor_device.py:load_registers:line 149:Getting available registers on device
05:45:04 PM:INFO:stage_model:stage_model.py:configure_device:line 81:Configuring registers with default values (can be modified in config file)
05:45:04 PM:INFO:move_worker:move_worker.py:goto_origin:line 194:(move to origin): Moving stage to origin
05:45:04 PM:INFO:move_worker:move_worker.py:move_relative:line 336:(move relative): Moved motors by steps specified in [0, -27200, -27200, 0]
05:45:07 PM:INFO:move_worker:move_worker.py:move_relative:line 336:(move relative): Moved motors by steps specified in [-27200, 0, 0, 0]
05:45:10 PM:INFO:move_worker:move_worker.py:move_relative:line 336:(move relative): Moved motors by steps specified in [0, 0, 0, -27200]
05:45:12 PM:INFO:stage_model:stage_model.py:configure_device:line 87:Zeroed all motor positions
05:45:12 PM:INFO:stage_model:stage_model.py:configure_device:line 88:Finished configuring stepper motor device
#############################################################
05:45:12 PM:INFO:Foraging:Foraging.py:_StopPhotometry:line 3428:Checking that photometry is not running
05:45:12 PM:INFO:Foraging:Foraging.py:_StopPhotometry:line 3438:Photometry excitation stopped
05:45:12 PM:INFO:Foraging:Foraging.py:_load_most_recent_rig_json:line 1534:Found existing rig.json: rig_446-8-D_2024-11-19_11_21_56.json
05:45:12 PM:INFO:Foraging:Foraging.py:_ReconnectBonsai:line 971:attempting to close bonsai connection
05:45:15 PM:INFO:Foraging:Foraging.py:_ReconnectBonsai:line 979:bonsai connection closed
05:45:15 PM:INFO:Foraging:Foraging.py:_ReconnectBonsai:line 981:attempting to restart bonsai
05:45:15 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1371:Trying to connect to already running Bonsai
05:45:15 PM:INFO:Foraging:Foraging.py:_ConnectOSC:line 1418:connecting to GUI and Bonsai through OSC
05:45:15 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1378:Connected to already running Bonsai
05:45:15 PM:INFO:Foraging:Foraging.py:_InitializeBonsai:line 1379:Bonsai started successfully
05:45:15 PM:INFO:Foraging:Foraging.py:__init__:line 219:Start up complete
05:45:16 PM:INFO:aws_util:aws_util.py:import_df_from_s3:line 115:Dataframe imported from s3://aind-behavior-data/foraging_nwb_bonsai_processed/df_sessions.pkl, len(df) = 5621
05:45:16 PM:INFO:aws_util:aws_util.py:import_df_from_s3:line 115:Dataframe imported from s3://aind-behavior-data/foraging_auto_training/df_manager_447_demo.pkl, len(df) = 4165
05:45:16 PM:INFO:aws_util:aws_util.py:download_dir_from_s3:line 140:134 objects downloaded from s3://aind-behavior-data/foraging_auto_training/saved_curriculums/ to C:\Users\svc_aind_behavior/capsule/scratch/saved_curriculums/
05:45:16 PM:INFO:curriculum_manager:curriculum_manager.py:download_curriculums:line 162:Found 26 curriculums in C:\Users\svc_aind_behavior/capsule/scratch/saved_curriculums/
05:45:17 PM:INFO:aws_util:aws_util.py:download_dir_from_s3:line 140:134 objects downloaded from s3://aind-behavior-data/foraging_auto_training/saved_curriculums/ to C:\Users\svc_aind_behavior/.aind_auto_train/curriculum_manager/
05:45:17 PM:INFO:curriculum_manager:curriculum_manager.py:download_curriculums:line 162:Found 26 curriculums in C:\Users\svc_aind_behavior/.aind_auto_train/curriculum_manager/
05:45:17 PM:INFO:Dialogs:Dialogs.py:update_auto_train_fields:line 2273:No entry found in df_training_manager for subject_id: 0
05:45:17 PM:INFO:Dialogs:Dialogs.py:_override_curriculum_clicked:line 2621:Override stage clicked: state=2
05:45:17 PM:INFO:Dialogs:Dialogs.py:_update_stage_to_apply:line 2647:Current stage to apply: unknown training stage @unknown curriculum
05:46:42 PM:INFO:Dialogs:Dialogs.py:_start_preview:line 1213:Camera is on
05:49:57 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ManualLeftWaterStartTime', 12003297.868, '127.0.0.1', 4033]
05:49:57 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftRewardDeliveryTimeHarp', 12003297.871488, '127.0.0.1', 4033]
05:50:02 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ManualLeftWaterStartTime', 12003302.774976, '127.0.0.1', 4033]
05:50:02 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftRewardDeliveryTimeHarp', 12003302.77648, '127.0.0.1', 4033]
05:50:03 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ManualRightWaterStartTime', 12003303.476992, '127.0.0.1', 4033]
05:50:03 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightRewardDeliveryTimeHarp', 12003303.479488, '127.0.0.1', 4033]
05:50:08 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ManualRightWaterStartTime', 12003308.084992, '127.0.0.1', 4033]
05:50:08 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RightRewardDeliveryTimeHarp', 12003308.086496, '127.0.0.1', 4033]
06:16:45 PM:INFO:Foraging:Foraging.py:_StopCurrentSession:line 3581:Stopping current trials
06:17:26 PM:INFO:Foraging:Foraging.py:_Open:line 2948:Quick load success: C:\behavior_data\446-8-D\756567\behavior_756567_2024-11-22_15-37-31\behavior\756567_2024-11-22_15-37-31.json
06:17:26 PM:INFO:Foraging:Foraging.py:_NewSession:line 3491:New Session pressed
06:17:26 PM:INFO:Foraging:Foraging.py:_NewSession:line 3523:Resetting session run flag
06:17:26 PM:INFO:Foraging:Foraging.py:_StopPhotometry:line 3428:Checking that photometry is not running
06:17:26 PM:INFO:Foraging:Foraging.py:_StopPhotometry:line 3438:Photometry excitation stopped
06:17:26 PM:INFO:Foraging:Foraging.py:_NewSession:line 3561:New Session complete
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1863:processing parameter changes
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: DelayMin, 0.0 -> 1.0
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: DelayBeta, 1.0 -> 0.0
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: RightValue, 0.03 -> 0.033
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: LeftValue, 0.03 -> 0.023
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: LeftValue_volume, 3.0 -> 2.0
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: ID, 0.0 -> 756567.0
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: DelayMin, 0.0 -> 1.0
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: DelayBeta, 1.0 -> 0.0
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: RightValue, 0.03 -> 0.033
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: LeftValue, 0.03 -> 0.023
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: LeftValue_volume, 3.0 -> 2.0
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: GiveWaterL, 0.03 -> 0.023
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: GiveWaterR, 0.03 -> 0.033
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: GiveWaterL_volume, 3.0 -> 2.0
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1863:processing parameter changes
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1863:processing parameter changes
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: BlockMax, 60.0 -> 35.0
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: ITIMax, 8.0 -> 30.0
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: ITIBeta, 2.0 -> 3.0
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: BlockMax, 60.0 -> 35.0
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: ITIMax, 8.0 -> 30.0
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: ITIBeta, 2.0 -> 3.0
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1863:processing parameter changes
06:17:26 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1863:processing parameter changes
06:17:27 PM:INFO:Foraging:Foraging.py:_check_drop_frames:line 682:
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1863:processing parameter changes
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: Ignored, 100.0 -> 7.0
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: BlockBeta, 20.0 -> 10.0
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: StopIgnores, 20.0 -> 25.0
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: MaxTime, 120.0 -> 75.0
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: Multiplier, 0.8 -> 0.5
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: RightValue_volume, 3.0 -> 2.0
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: RightValue, 0.033 -> 0.022
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: Unrewarded, 200.0 -> 7.0
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: Ignored, 100.0 -> 7.0
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: BlockBeta, 20.0 -> 10.0
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: StopIgnores, 20.0 -> 25.0
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: MaxTime, 120.0 -> 75.0
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: Multiplier, 0.8 -> 0.5
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: RightValue_volume, 3.0 -> 2.0
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: RightValue, 0.033 -> 0.022
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: Unrewarded, 200.0 -> 7.0
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: GiveWaterR, 0.033 -> 0.022
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: GiveWaterR_volume, 3.0 -> 2.0
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: lineEdit_init_stage_pos_z, 12.0 -> 10.0
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: lineEdit_y1, 0.0 -> 25.9
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: lineEdit_z, 0.0 -> 20.6
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: lineEdit_y2, 0.0 -> 25.9
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: lineEdit_x, 0.0 -> 16.585
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: lineEdit_step_size, 1.0 -> 0.1
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: SuggestedWater,  -> 0.855
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: BaseWeight,  -> 24.3
06:17:27 PM:INFO:Foraging:Foraging.py:keyPressEvent:line 1922:Changing parameter: TargetWeight,  -> 20.655
06:17:27 PM:INFO:Dialogs:Dialogs.py:update_auto_train_lock:line 2825:AutoTrain disengaged!
06:17:27 PM:INFO:curriculum_manager:curriculum_manager.py:get_curriculum:line 131:Loaded a DynamicForagingCurriculum model from 'Uncoupled Baiting_curriculum_v2.3_schema_v1.0.json'.
06:17:27 PM:INFO:Dialogs:Dialogs.py:_override_curriculum_clicked:line 2621:Override stage clicked: state=0
06:17:27 PM:INFO:Dialogs:Dialogs.py:_update_curriculum_diagrams:line 2567:Selected curriculum: {'curriculum_name': 'Uncoupled Baiting', 'curriculum_version': '2.3', 'curriculum_schema_version': '1.0', 'curriculum_description': '2024-08-16 max_len = 75 mins; decrease finished trial criterion'}
06:17:27 PM:INFO:curriculum_manager:curriculum_manager.py:get_curriculum:line 131:Loaded a DynamicForagingCurriculum model from 'Uncoupled Baiting_curriculum_v2.3_schema_v1.0.json'.
06:17:27 PM:INFO:Dialogs:Dialogs.py:_update_stage_to_apply:line 2647:Current stage to apply: GRADUATED @Uncoupled Baiting (v2.3@1.0)
06:17:27 PM:INFO:Dialogs:Dialogs.py:_update_stage_to_apply:line 2647:Current stage to apply: GRADUATED @Uncoupled Baiting (v2.3@1.0)
06:17:27 PM:INFO:Dialogs:Dialogs.py:_update_curriculum_diagrams:line 2567:Selected curriculum: {'curriculum_name': 'Uncoupled Baiting', 'curriculum_version': '2.3', 'curriculum_schema_version': '1.0', 'curriculum_description': '2024-08-16 max_len = 75 mins; decrease finished trial criterion'}
06:17:27 PM:INFO:curriculum_manager:curriculum_manager.py:get_curriculum:line 131:Loaded a DynamicForagingCurriculum model from 'Uncoupled Baiting_curriculum_v2.3_schema_v1.0.json'.
06:17:27 PM:INFO:Dialogs:Dialogs.py:update_auto_train_lock:line 2774:AutoTrain engaged! GRADUATED @ Uncoupled Baiting (v2.3@1.0)
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2905: Widget "training_stage" not found. skipped...
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2897:Task is set to Uncoupled Baiting
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2905: Widget "task_url" not found. skipped...
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2905: Widget "task_schema_version" not found. skipped...
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2905: Widget "description" not found. skipped...
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2917: Widget "BaseRewardSum" has been disabled by the GUI. skipped...
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2917: Widget "RewardFamily" has been disabled by the GUI. skipped...
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2917: Widget "RewardPairsN" has been disabled by the GUI. skipped...
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2952:UncoupledReward is set to 0.1, 0.4, 0.7
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2952:Randomness is set to Exponential
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2952:BlockMin is set to 20
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2952:BlockMax is set to 35
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2917: Widget "BlockBeta" has been disabled by the GUI. skipped...
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2917: Widget "BlockMinReward" has been disabled by the GUI. skipped...
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2952:DelayMin is set to 1.0
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2952:DelayMax is set to 1.0
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2952:DelayBeta is set to 0.0
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2952:RewardDelay is set to 0.0
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2952:AutoReward is set to False
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2917: Widget "AutoWaterType" has been disabled by the GUI. skipped...
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2917: Widget "Multiplier" has been disabled by the GUI. skipped...
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2917: Widget "Unrewarded" has been disabled by the GUI. skipped...
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2917: Widget "Ignored" has been disabled by the GUI. skipped...
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2952:ITIMin is set to 1.0
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2952:ITIMax is set to 30.0
06:17:27 PM:INFO:Dialogs:Dialogs.py:_set_training_parameters:line 2952:ITIBeta is set to 3.0
_____SKIPPING_____
09:24:27 PM:INFO:Foraging:Foraging.py:_GetWaterCalibration:line 1064:Loaded Water Calibration
09:24:27 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 88:processing:C:\behavior_data\446-8-D\757214\behavior_757214_2024-11-23_20-10-09
09:24:27 PM:INFO:GenerateMetadata:GenerateMetadata.py:_handle_edge_cases:line 361:Cannot log reward delivery in session metadata - missing fields
09:24:27 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_reward_delivery:line 1192:No reward delivery metadata found!
09:24:27 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_opto_calibration:line 1027:No opto calibration results detected!
09:24:27 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 865:No ephys data stream detected!
09:24:27 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 584:No photometry data stream detected!
09:24:27 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 114:Session metadata generated successfully: True
09:24:27 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 115:Rig metadata generated successfully: True
09:24:27 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_reward_delivery:line 1192:No reward delivery metadata found!
09:24:27 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_opto_calibration:line 1027:No opto calibration results detected!
09:24:27 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 865:No ephys data stream detected!
09:24:27 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 584:No photometry data stream detected!
09:24:28 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/DelayStartTime', 12016168.368992, '127.0.0.1', 4032]
09:24:28 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 12016168.370496, '127.0.0.1', 4032]
09:24:29 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTime', 12016169.352, '127.0.0.1', 4032]
09:24:29 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 12016169.354496, '127.0.0.1', 4032]
09:24:29 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTimeSoundCard', 12016169.3552, 4, '127.0.0.1']
09:24:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcomeTime', 12016170.352, '127.0.0.1', 4032]
09:24:30 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcome', 'NoResponse', '127.0.0.1', 4032]
09:24:33 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialEndTime', 12016173.368992, '127.0.0.1', 4032]
09:24:33 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 12016173.370496, '127.0.0.1', 4032]
09:24:33 PM:WARNING:MyFunctions:MyFunctions.py:_GenerateATrial:line 172:
09:24:33 PM:WARNING:MyFunctions:MyFunctions.py:_CheckStop:line 1191:
09:24:33 PM:INFO:Foraging:Foraging.py:_StartTrialLoop:line 4141:Current trial: 466
09:24:33 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialStartTime', 12016173.386976, '127.0.0.1', 4032]
09:24:33 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ITIStartTimeHarp', 12016173.391488, '127.0.0.1', 4032]
09:24:33 PM:WARNING:warnings:warnings.py:_showwarnmsg:line 110:C:\Users\svc_aind_behavior\Documents\GitHub\dynamic-foraging-task\src\foraging_gui\Visualization.py:160: RuntimeWarning: Mean of empty slice
  RewardedHistoryF[i+1-self.kernel_size]=np.nanmean(RewardedHistoryT[i+1-self.kernel_size:i+1])

09:24:33 PM:INFO:Foraging:Foraging.py:_Save:line 2441:Saving current session, ForceSave=1
09:24:33 PM:INFO:Foraging:Foraging.py:_GetWaterCalibration:line 1064:Loaded Water Calibration
09:24:33 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 88:processing:C:\behavior_data\446-8-D\757214\behavior_757214_2024-11-23_20-10-09
09:24:33 PM:INFO:GenerateMetadata:GenerateMetadata.py:_handle_edge_cases:line 361:Cannot log reward delivery in session metadata - missing fields
09:24:33 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_reward_delivery:line 1192:No reward delivery metadata found!
09:24:33 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_opto_calibration:line 1027:No opto calibration results detected!
09:24:33 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 865:No ephys data stream detected!
09:24:33 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 584:No photometry data stream detected!
09:24:33 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 114:Session metadata generated successfully: True
09:24:33 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 115:Rig metadata generated successfully: True
09:24:33 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_reward_delivery:line 1192:No reward delivery metadata found!
09:24:33 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_opto_calibration:line 1027:No opto calibration results detected!
09:24:33 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 865:No ephys data stream detected!
09:24:33 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 584:No photometry data stream detected!
09:24:34 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/DelayStartTime', 12016174.501984, '127.0.0.1', 4032]
09:24:34 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 12016174.50448, '127.0.0.1', 4032]
09:24:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTime', 12016175.485984, '127.0.0.1', 4032]
09:24:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 12016175.487488, '127.0.0.1', 4032]
09:24:35 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTimeSoundCard', 12016175.488096, 4, '127.0.0.1']
09:24:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcomeTime', 12016176.485984, '127.0.0.1', 4032]
09:24:36 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcome', 'NoResponse', '127.0.0.1', 4032]
09:24:39 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialEndTime', 12016179.501984, '127.0.0.1', 4032]
09:24:39 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 12016179.50448, '127.0.0.1', 4032]
09:24:39 PM:WARNING:MyFunctions:MyFunctions.py:_GenerateATrial:line 172:
09:24:39 PM:WARNING:MyFunctions:MyFunctions.py:_CheckStop:line 1191:
09:24:39 PM:INFO:Foraging:Foraging.py:_StartTrialLoop:line 4141:Current trial: 467
09:24:39 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialStartTime', 12016179.520992, '127.0.0.1', 4032]
09:24:39 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ITIStartTimeHarp', 12016179.52448, '127.0.0.1', 4032]
09:24:39 PM:WARNING:warnings:warnings.py:_showwarnmsg:line 110:C:\Users\svc_aind_behavior\Documents\GitHub\dynamic-foraging-task\src\foraging_gui\Visualization.py:160: RuntimeWarning: Mean of empty slice
  RewardedHistoryF[i+1-self.kernel_size]=np.nanmean(RewardedHistoryT[i+1-self.kernel_size:i+1])

09:24:39 PM:INFO:Foraging:Foraging.py:_Save:line 2441:Saving current session, ForceSave=1
09:24:39 PM:INFO:Foraging:Foraging.py:_GetWaterCalibration:line 1064:Loaded Water Calibration
09:24:39 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 88:processing:C:\behavior_data\446-8-D\757214\behavior_757214_2024-11-23_20-10-09
09:24:39 PM:INFO:GenerateMetadata:GenerateMetadata.py:_handle_edge_cases:line 361:Cannot log reward delivery in session metadata - missing fields
09:24:39 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_reward_delivery:line 1192:No reward delivery metadata found!
09:24:39 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_opto_calibration:line 1027:No opto calibration results detected!
09:24:39 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 865:No ephys data stream detected!
09:24:39 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 584:No photometry data stream detected!
09:24:39 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 114:Session metadata generated successfully: True
09:24:39 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 115:Rig metadata generated successfully: True
09:24:39 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_reward_delivery:line 1192:No reward delivery metadata found!
09:24:39 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_opto_calibration:line 1027:No opto calibration results detected!
09:24:39 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 865:No ephys data stream detected!
09:24:39 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 584:No photometry data stream detected!
09:24:48 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/DelayStartTime', 12016188.568992, '127.0.0.1', 4032]
09:24:48 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 12016188.570496, '127.0.0.1', 4032]
09:24:49 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTime', 12016189.552, '127.0.0.1', 4032]
09:24:49 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 12016189.554496, '127.0.0.1', 4032]
09:24:49 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTimeSoundCard', 12016189.555264, 4, '127.0.0.1']
09:24:49 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 12016189.669408, '127.0.0.1', 4033]
09:24:49 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcomeTime', 12016189.668992, '127.0.0.1', 4032]
09:24:49 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcome', 'ErrorLeft', '127.0.0.1', 4032]
09:24:52 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialEndTime', 12016192.668992, '127.0.0.1', 4032]
09:24:52 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 12016192.670496, '127.0.0.1', 4032]
09:24:52 PM:WARNING:MyFunctions:MyFunctions.py:_GenerateATrial:line 172:
09:24:52 PM:WARNING:MyFunctions:MyFunctions.py:_CheckStop:line 1191:
09:24:52 PM:INFO:Foraging:Foraging.py:_StartTrialLoop:line 4141:Current trial: 468
09:24:52 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialStartTime', 12016192.686976, '127.0.0.1', 4032]
09:24:52 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ITIStartTimeHarp', 12016192.690496, '127.0.0.1', 4032]
09:24:52 PM:WARNING:warnings:warnings.py:_showwarnmsg:line 110:C:\Users\svc_aind_behavior\Documents\GitHub\dynamic-foraging-task\src\foraging_gui\Visualization.py:160: RuntimeWarning: Mean of empty slice
  RewardedHistoryF[i+1-self.kernel_size]=np.nanmean(RewardedHistoryT[i+1-self.kernel_size:i+1])

09:24:52 PM:INFO:Foraging:Foraging.py:_Save:line 2441:Saving current session, ForceSave=1
09:24:52 PM:INFO:Foraging:Foraging.py:_GetWaterCalibration:line 1064:Loaded Water Calibration
09:24:52 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 88:processing:C:\behavior_data\446-8-D\757214\behavior_757214_2024-11-23_20-10-09
09:24:52 PM:INFO:GenerateMetadata:GenerateMetadata.py:_handle_edge_cases:line 361:Cannot log reward delivery in session metadata - missing fields
09:24:52 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_reward_delivery:line 1192:No reward delivery metadata found!
09:24:52 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_opto_calibration:line 1027:No opto calibration results detected!
09:24:52 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 865:No ephys data stream detected!
09:24:52 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 584:No photometry data stream detected!
09:24:52 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 114:Session metadata generated successfully: True
09:24:52 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 115:Rig metadata generated successfully: True
09:24:52 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_reward_delivery:line 1192:No reward delivery metadata found!
09:24:52 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_opto_calibration:line 1027:No opto calibration results detected!
09:24:52 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 865:No ephys data stream detected!
09:24:52 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 584:No photometry data stream detected!
09:24:53 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/DelayStartTime', 12016193.768992, '127.0.0.1', 4032]
09:24:53 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 12016193.770496, '127.0.0.1', 4032]
09:24:54 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTime', 12016194.752992, '127.0.0.1', 4032]
09:24:54 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 12016194.754496, '127.0.0.1', 4032]
09:24:54 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTimeSoundCard', 12016194.7552, 4, '127.0.0.1']
09:24:55 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcomeTime', 12016195.752992, '127.0.0.1', 4032]
09:24:55 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcome', 'NoResponse', '127.0.0.1', 4032]
09:24:58 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialEndTime', 12016198.768992, '127.0.0.1', 4032]
09:24:58 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 12016198.771488, '127.0.0.1', 4032]
09:24:58 PM:WARNING:MyFunctions:MyFunctions.py:_GenerateATrial:line 172:
09:24:58 PM:WARNING:MyFunctions:MyFunctions.py:_CheckStop:line 1191:
09:24:58 PM:INFO:Foraging:Foraging.py:_StartTrialLoop:line 4141:Current trial: 469
09:24:58 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialStartTime', 12016198.788, '127.0.0.1', 4032]
09:24:58 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ITIStartTimeHarp', 12016198.791488, '127.0.0.1', 4032]
09:24:58 PM:WARNING:warnings:warnings.py:_showwarnmsg:line 110:C:\Users\svc_aind_behavior\Documents\GitHub\dynamic-foraging-task\src\foraging_gui\Visualization.py:160: RuntimeWarning: Mean of empty slice
  RewardedHistoryF[i+1-self.kernel_size]=np.nanmean(RewardedHistoryT[i+1-self.kernel_size:i+1])

09:24:58 PM:INFO:Foraging:Foraging.py:_Save:line 2441:Saving current session, ForceSave=1
09:24:58 PM:INFO:Foraging:Foraging.py:_GetWaterCalibration:line 1064:Loaded Water Calibration
09:24:58 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 88:processing:C:\behavior_data\446-8-D\757214\behavior_757214_2024-11-23_20-10-09
09:24:58 PM:INFO:GenerateMetadata:GenerateMetadata.py:_handle_edge_cases:line 361:Cannot log reward delivery in session metadata - missing fields
09:24:58 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_reward_delivery:line 1192:No reward delivery metadata found!
09:24:58 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_opto_calibration:line 1027:No opto calibration results detected!
09:24:58 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 865:No ephys data stream detected!
09:24:58 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 584:No photometry data stream detected!
09:24:58 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 114:Session metadata generated successfully: True
09:24:58 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 115:Rig metadata generated successfully: True
09:24:58 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_reward_delivery:line 1192:No reward delivery metadata found!
09:24:58 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_opto_calibration:line 1027:No opto calibration results detected!
09:24:58 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 865:No ephys data stream detected!
09:24:58 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 584:No photometry data stream detected!
09:24:59 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/DelayStartTime', 12016199.885984, '127.0.0.1', 4032]
09:24:59 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 12016199.887488, '127.0.0.1', 4032]
09:25:00 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTime', 12016200.868992, '127.0.0.1', 4032]
09:25:00 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTimeSoundCard', 12016200.871104, 4, '127.0.0.1']
09:25:00 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 12016200.871488, '127.0.0.1', 4032]
09:25:01 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcomeTime', 12016201.868992, '127.0.0.1', 4032]
09:25:01 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcome', 'NoResponse', '127.0.0.1', 4032]
09:25:04 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialEndTime', 12016204.885984, '127.0.0.1', 4032]
09:25:04 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 12016204.887488, '127.0.0.1', 4032]
09:25:04 PM:WARNING:MyFunctions:MyFunctions.py:_GenerateATrial:line 172:
09:25:04 PM:WARNING:MyFunctions:MyFunctions.py:_CheckStop:line 1191:
09:25:04 PM:INFO:Foraging:Foraging.py:_StartTrialLoop:line 4141:Current trial: 470
09:25:04 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialStartTime', 12016204.904, '127.0.0.1', 4032]
09:25:04 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/ITIStartTimeHarp', 12016204.907488, '127.0.0.1', 4032]
09:25:04 PM:WARNING:warnings:warnings.py:_showwarnmsg:line 110:C:\Users\svc_aind_behavior\Documents\GitHub\dynamic-foraging-task\src\foraging_gui\Visualization.py:160: RuntimeWarning: Mean of empty slice
  RewardedHistoryF[i+1-self.kernel_size]=np.nanmean(RewardedHistoryT[i+1-self.kernel_size:i+1])

09:25:04 PM:INFO:Foraging:Foraging.py:_Save:line 2441:Saving current session, ForceSave=1
09:25:04 PM:INFO:Foraging:Foraging.py:_GetWaterCalibration:line 1064:Loaded Water Calibration
09:25:04 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 88:processing:C:\behavior_data\446-8-D\757214\behavior_757214_2024-11-23_20-10-09
09:25:04 PM:INFO:GenerateMetadata:GenerateMetadata.py:_handle_edge_cases:line 361:Cannot log reward delivery in session metadata - missing fields
09:25:04 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_reward_delivery:line 1192:No reward delivery metadata found!
09:25:04 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_opto_calibration:line 1027:No opto calibration results detected!
09:25:05 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 865:No ephys data stream detected!
09:25:05 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 584:No photometry data stream detected!
09:25:05 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 114:Session metadata generated successfully: True
09:25:05 PM:INFO:GenerateMetadata:GenerateMetadata.py:__init__:line 115:Rig metadata generated successfully: True
09:25:05 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_reward_delivery:line 1192:No reward delivery metadata found!
09:25:05 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_opto_calibration:line 1027:No opto calibration results detected!
09:25:05 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ephys_stream:line 865:No ephys data stream detected!
09:25:05 PM:INFO:GenerateMetadata:GenerateMetadata.py:_get_ophys_stream:line 584:No photometry data stream detected!
09:25:06 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/DelayStartTime', 12016206.668992, '127.0.0.1', 4032]
09:25:06 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 12016206.671488, '127.0.0.1', 4032]
09:25:07 PM:INFO:bias_indicator:bias_indicator.py:calculate_bias:line 139:Bias: -0.0034000260291503492 Trial Count: 469
09:25:07 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTime', 12016207.652992, '127.0.0.1', 4032]
09:25:07 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 12016207.654496, '127.0.0.1', 4032]
09:25:07 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/GoCueTimeSoundCard', 12016207.655392, 4, '127.0.0.1']
09:25:08 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcomeTime', 12016208.652992, '127.0.0.1', 4032]
09:25:08 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/RewardOutcome', 'NoResponse', '127.0.0.1', 4032]
09:25:11 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/TrialEndTime', 12016211.668992, '127.0.0.1', 4032]
09:25:11 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/BehaviorEvent', 12016211.671488, '127.0.0.1', 4032]
09:25:11 PM:WARNING:MyFunctions:MyFunctions.py:_GenerateATrial:line 172:
09:25:11 PM:WARNING:MyFunctions:MyFunctions.py:_CheckStop:line 1191:Stop because running time exceeds or equals: 75m
09:30:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 913:Percentage of left lick intervals under 100 ms is 0.79%.
09:30:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 919:Percentage of right lick intervals under 100 ms is 0.46%.
09:30:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 926:Percentage of right and left lick intervals under 100 ms is 0.68%.
09:30:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 951:Percentage of cross side lick intervals under 100 ms is 1.85%.
09:40:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 913:Percentage of left lick intervals under 100 ms is 0.79%.
09:40:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 919:Percentage of right lick intervals under 100 ms is 0.46%.
09:40:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 926:Percentage of right and left lick intervals under 100 ms is 0.68%.
09:40:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 951:Percentage of cross side lick intervals under 100 ms is 1.85%.
09:50:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 913:Percentage of left lick intervals under 100 ms is 0.79%.
09:50:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 919:Percentage of right lick intervals under 100 ms is 0.46%.
09:50:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 926:Percentage of right and left lick intervals under 100 ms is 0.68%.
09:50:09 PM:INFO:MyFunctions:MyFunctions.py:calculate_inter_lick_intervals:line 951:Percentage of cross side lick intervals under 100 ms is 1.85%.
09:53:52 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 12017932.181824, '127.0.0.1', 4033]
09:53:52 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 12017932.282176, '127.0.0.1', 4033]
09:53:58 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 12017938.254944, '127.0.0.1', 4033]
09:53:58 PM:INFO:rigcontrol:rigcontrol.py:msg_handler:line 56:['/LeftLickTime', 12017938.427584, '127.0.0.1', 4033]
09:55:04 PM:INFO:Foraging:Foraging.py:_Start:line 3912:Start button pressed: ending trial loop
09:55:04 PM:INFO:Foraging:Foraging.py:_StartTrialLoop:line 4122:ending trial loop
09:55:07 PM:ERROR:Foraging:Foraging.py:_StartTrialLoop:line 4266:trial stalled 5 minutes, user stopped trials. ANewTrial:1,Start:False,finish_Timer:1

ellahiltonvano commented 1 day ago

@XX-Yin @micahwoodard this is happening frequently, is there a plan to address this?

micahwoodard commented 1 day ago

Another possible reason is the GUI stopped after exceeding the time limit (75m by default), but the stall function will keep running to get this "5m stall error information". We should stop the stall function in this edge case.

@XX-Yin did you ever follow up with this thought? I implemented the threading fix for the bias so I don't think it's coming from there

micahwoodard commented 1 day ago

@ellahiltonvano do you know what the behavior of this bug is? Does the gui crash or does a warning just pop up?

ellahiltonvano commented 1 day ago

@micahwoodard I suspect its more related to a crash since I haven't seen any issues about a warning, but can't say for sure

micahwoodard commented 1 day ago

Yeah and it seems like the logs stop after the error which indicates a crash to me. Hmmmm we should probably bump up this priority if it's crashing with this known bug pretty regularly. I can take a closer look this week. Can't promise I'll get too far since I'm working remotely but I can try and maybe hand it off to someone!

XX-Yin commented 1 day ago

Another possible reason is the GUI stopped after exceeding the time limit (75m by default), but the stall function will keep running to get this "5m stall error information". We should stop the stall function in this edge case

I submitted a new issue specific for this question. Users don't report the "5m stall", I speculate that at least some cases are related to this issue.

950

I doubt this is related to this issue. You can see the 75m indication. Also, the user doesn't seem to notice the crash.