search

BouchardLab / nsds_lab_to_nwb

Python package to convert NSDS Lab data to NWB files.
https://nsds-lab-to-nwb.readthedocs.io/en/latest/
0 stars 4 forks source link

clarify value of mark_threshold for wn2 #102

Closed jihyunbak closed 3 years ago

jihyunbak commented 3 years ago

It seems we need a lower mark_threshold to detect all wn2 stim onsets for the test block RVG16_B01. Should figure out why … is this just for this block, or should we fix the value in the stim metadata wn2.yaml?

The temporary code (for setting the alternative mark threshold value) is commented out:

https://github.com/BouchardLab/nsds_lab_to_nwb/blob/bf693660eda576fbde1ca4d7124c1b2e4382d44b/nsds_lab_to_nwb/components/stimulus/tokenizers/wn_tokenizer.py#L70-L86

jthermiz commented 3 years ago

@jihyunbak I'm not sure why we need to threshold the marks. The marks are already detected by TDT software and can be read out by TDTReader like this:

#%% imports

from nsds_lab_to_nwb.tools.tdt.tdt_reader import TDTReader

#%% parameters

path = '/home/jhermiz/data/aer/RVG16/RVG16_B01'

#%% load tdt

tdt_obj = TDTReader(path)
evs = tdt_obj.get_events()

# %%
print(len(evs))
print(evs)

I ran this can get all 60 marks as expected space 1 second apart. Here are the event times in seconds relative to when the recording started

[13.3238784  14.31953408 15.31957248 16.31956992 17.31956736 18.3195648
 19.31956224 20.31955968 21.31959808 22.31959552 23.31959296 24.3195904
 25.31958784 26.31958528 27.31962368 28.31962112 29.31961856 30.319616
 31.31961344 32.31961088 33.31964928 34.31964672 35.31964416 36.3196416
 37.31963904 38.31963648 39.31967488 40.31967232 41.31966976 42.3196672
 43.31966464 44.31966208 45.31970048 46.31969792 47.31969536 48.3196928
 49.31969024 50.31972864 51.31972608 52.31972352 53.31972096 54.3197184
 55.31971584 56.31975424 57.31975168 58.31974912 59.31974656 60.319744
 61.31974144 62.31977984 63.31977728 64.31977472 65.31977216 66.3197696
 67.319808   68.31980544 69.31980288 70.31980032 71.31979776 72.3197952 ]
jthermiz commented 3 years ago

@vngutierrez FYI

jihyunbak commented 3 years ago

@jthermiz Thanks a lot - I was not aware of this! (Likely you've already mentioned and I forgot…) We should definitely use this in the TDT pipeline.

jihyunbak commented 3 years ago

@VBaratham could you also look at this mark threshold issue? The metadata file wn2.yaml (content taken from mars) says mark_threshold = 0.3. But for this particular test block with WN2 stimuli, using a threshold of 0.3 detected only 59 out of 60 events; using an (arbitrary) threshold of 0.1 detected all 60 events.

Of course we can bypass this using the TDT data as @jthermiz suggested, but would like to understand what's happening here if possible...

VBaratham commented 3 years ago

I agree that if marks are detected properly by TDT we should use that data.

I don't think there's anything special about the mark thresholds we used in the past, just numbers that seemed to work. So it wouldn't be unreasonable to use 0.1 instead of 0.3. But I'm not sure I understand why 0.3 doesn't work. From issue #76 it looks like mark_threshold of 0.3 fails on a block from R56 as well as one of the new blocks. That is puzzling to me, as I don't think I encountered this using Mars, and I definitely processed R56. It's also kind of weird that the mark for one trial would have a smaller amplitude than all the others... If we want to dig into it, it may help to look at a plot of one of these recorded mark tracks.

jihyunbak commented 3 years ago

Thanks for the input @VBaratham!