VBN sessions table 'session number' column doesn't reflect failed sessions

[corbennett]

Describe the bug In the latest VBN sessions metadata table here: \allen\programs\mindscope\workgroups\np-behavior\vbn_data_release\metadata_220527\ecephys_sessions.csv, there are several mice for which the second recording day is misrepresented as 'session_number' 1. For all of these mice, the first ephys session was failed (and is not part of the release). However, we would still like to indicate which recording day (1 or 2) these sessions come from. So even though there is only one session in the release for these mice, the session number should be 2.

Here are the mice: 524760 546512 548465

[danielsf]

@corbennett The way we have been generating these metadata tables, we have been just passing in a list of the passed ecephys_session_ids and running a bunch of LIMS queries with those as the input to the WHERE clauses.

I had hoped that we could address this issue by:

1) finding all of the donor_ids associated with the passed ecephys_session_ids 2) getting all of the ecephys_sessions associated with those donor_ids (which will now contained passed and failed sessions) 3) processing ecephys_sessions.csv as normal 4) dropping failed sessions

But I see that there are more than 2 ecephys sessions associated with each donor_id in the LIMS database, which means that this approach will scrape up many more than the "day 1" and "day 2" sessions for VBN.

Is there any other column in the LIMS database that I can use to only select only the sessions associated with this data release, or does that information only exist in your spreadsheet?

[danielsf]

I think the way to fix this is to alter the metadata_writer input JSON schema so that ecephys_session_id_list encompasses all of the sessions in the release, even the failed ones. We can then add a field failed_ecephys_session_id_list that lists sessions to be failed. The inclusive list will be used when constructing the mouse's history of sessions, exposure to images, etc. There will then be a final step in the metadata_writer to drop the failed sessions from ecephys_sessions.csv.

[danielsf]

addressed by #2455