Add flag to make eyetracking data optional for BehaviorOphysWriteNwb

[wbwakeman]

Three sessions for the March Visual Behavior release have good behavior and ophys data, but some error occurred with eye tracking such that it is not possible to release that eye tracking data. We need to make NWB files for this data that skips the eye tracking part.

This issue is to add a flag (or some other reasonable mechanism) that will allow us to create an NWB file for this data. Essentially skipping the part that adds eye tracking data to the NWB file.

Note that we track whether the eye tracking is passed or failed in the warehouse so a user will have a way to determine which sessions do, or do not, have valid eye tracking data.

[wbwakeman]

How LIMS runs one of these jobs:

/allen/aibs/technology/conda/run_miniconda.sh /allen/aibs/technology/waynew/conda/allensdk_py36 \
python -m allensdk.brain_observatory.behavior.write_nwb \
--input_json /path/to/input.json \
 --output_json /path/to/output.json

Example input.json file:
/allen/programs/braintv/production/visualbehavior/prod5/specimen_1056093392/ophys_session_1084882685/ophys_experiment_1085000574/BEHAVIOR_OPHYS_WRITE_NWB_QUEUE_1085000574_input.json
(can copy to your location and change output_path to a location is writable for you)

Desired change is to run with a flag something like this:

/allen/aibs/technology/conda/run_miniconda.sh /allen/aibs/technology/waynew/conda/allensdk_py36 \
python -m allensdk.brain_observatory.behavior.write_nwb \
--skip_eye_tracking True \
--input_json /path/to/input.json \
 --output_json /path/to/output.json

And then double-check that the output file does not contain the acquisition/EyeTracking section

[danielsf]

@wbwakeman Under what version of the SDK did you last try to run this session?

When I try to run

BEHAVIOR_OPHYS_WRITE_NWB_QUEUE_1085000574_input.json

through write_nwb, I get the following marshmallow validation error

marshmallow.exceptions.ValidationError: {'session_data': {'imaging_plane_group': ['Missing data for required field.'], 'behavior_session_id': ['Missing data for required field.'], 'foraging_id': ['Missing data for required field.'], 'ophys_session_id': ['Missing data for required field.'], 'plane_group_count': ['Missing data for required field.'], 'eye_gaze_mapping_path': ['Unknown field.']}}

The file is indeed missing ophys_session_id. It also contains eye_gaze_mapping_path, which rc/2.8.0 doesn't expect. Is it possible that, in addition to lacking eye tracking data, this input.json is very old? Is there a more modern version of input.json that I can use to produce the bug at hand?

[danielsf]

It looks like eye_gaze_mapping_path is a part of the ecephys write_nwb schema, if that helps

[wbwakeman]

I grabbed one that was a whole 5 days old. A lot has changed in that time. How about this one?

/allen/programs/braintv/production/visualbehavior/prod0/specimen_830901424/ophys_session_872551243/ophys_experiment_873154932/BEHAVIOR_OPHYS_WRITE_NWB_QUEUE_NO_STATE_CHANGE_873154932_input.json

[danielsf]

Thanks. That one is running.

[danielsf]

@wbwakeman

I think I have a solution. Do you want to check that branch ticket/1942/dev does what you want.

The command is

 python -m allensdk.brain_observatory.behavior.write_nwb --input_json BEHAVIOR_OPHYS_WRITE_NWB_QUEUE_NO_STATE_CHANGE_873154932_input.json --output_json output.json --skip_eye_tracking True

[matchings]

@wbwakeman could you list which sessions are affected by this so that I know which ones to expect no eye tracking data for?

[wbwakeman]

@matchings
795953296 806456687 833631914