AllenInstitute / visual_behavior_analysis

Python package for analyzing behavioral data for Brain Observatory: Visual Behavior
Other
21 stars 6 forks source link

get filtered_behavior_session_table has wrong session_type information #723

Closed alexpiet closed 3 years ago

alexpiet commented 3 years ago

@matchings This is a distinct, but related issue to #722

Here, the problem is that the session type is simply WRONG for many sessions. I found these sessions by noticing there were sessions with ophys_session_id = NaN, yet had session_type = OPHYS_

t = loading.get_filtered_behavior_session_table(release_data_only=True)
bad_session_type = t[t.ophys_session_id.isnull() & t.session_type.str.startswith('OPHYS') &~t.session_type.str.startswith('OPHYS_0')]

Spot checking these sessions, when I load the session object the metadata and stimulus_presentations table are consistent with these sessions being training sessions EDIT: I think I had a bug in my code, and now checking these systematically, it looks like the metadata matches what is in the filtered table

alexpiet commented 3 years ago

debugging this, it could be an issue inherited from the SDK. However these sessions all have the correct session type, but just have a nan ophys_session_id

cache = BehaviorProjectCache.from_lims(manifest = pgt.loading.get_manifest_path())
behavior_sessions = cache.get_behavior_session_table()
b = cache.get_behavior_session_table()
bad_sessions = b[b.ophys_session_id.isnull()&b.session_type.str.startswith('OPHYS_1')]
bad_sessions = bad_sessions.query('equipment_name in ["MESO.1","CAM2P.3","CAM2P.4","CAM2P.5"]') #table of 116 sessions
alexpiet commented 3 years ago

Screen Shot 2021-03-08 at 9 57 47 AM

Checking the sessions obtained from the loading.get_filtered_behavior_session_table() function, these 5 have a mis-match

alexpiet commented 3 years ago

Repeating the session check with more information. Note that these 5 mis matches are a separate issue from the main problem I'm describing

Screen Shot 2021-03-08 at 10 27 39 AM

alexpiet commented 3 years ago

It appears all the sessions identified from loading.get_filtered_... are also identified from the sdk behavior_sessions table. This indicates this is a SDK issue. Therefore I am closing this issue.