search

AllenInstitute / AllenSDK

code for reading and processing Allen Institute for Brain Science data
https://allensdk.readthedocs.io/en/latest/
Other
335 stars 149 forks source link

Inconsistent handling of omitted stimuli in VBN ecephys and behavior-only sessions #2578

Closed corbennett closed 1 year ago

corbennett commented 1 year ago

Describe the bug Loading the same VBN session as an ecephys session or behavior-only session yields slightly different results for omitted stimuli in the stimulus presentations table.

For ecephys sessions, omitted stimuli have NaN for duration, but an integer for 'end_frame'. For behavior-only sessions they have a float for duration and a NaN for end frame.

To Reproduce @morriscb shared the latest behavior-only stim table with me. Not sure exactly how this was produced.

Expected behavior I think it would make sense to just use the non-NaN values for both columns.

morriscb commented 1 year ago

Hey @corbennett, what is the expected behavior you would like? As in, what null values should these columns be filled with then they are truly NaN or Null? The typing issue is a feature of pandas/numpy as NaN values are float. For duration, of an omitted column, should this just be the time between the two stimuli around it or are it's start/stop times available?

morriscb commented 1 year ago

Similar to what was mentioned in #2581 we can edit the behavior only session table to match the ecephys data, but will leave the already released data as is for this release.

corbennett commented 1 year ago

In the end (when we can fix the released data), we would want the following behavior for both ecephys and behavior only stim tables for the omitted stimulus rows:

end_frame = start_frame+15

Then duration should be calculated using the experiment frame times (taking the timestamp for the end frame and subtracting the timestamp for the start frame). This should come out to be very close to 0.25 seconds.