search

CYHSM / DeepInsight

A general framework for interpreting wide-band neural activity
MIT License
155 stars 33 forks source link

Questions regarding decoder target #7

Closed PierreOrhan closed 4 years ago

PierreOrhan commented 4 years ago

Thank you for this interesting paper and work! I had a question regarding the decoder here coded: Is it correct that your model takes as input wavelet powers for a temporal window "T=64 (corresponding to 2.13s)" (which are in fact down-sampled M=1000 times over the temporal dimension from the original wavelet powers matrix), and output one value for each behavioral variable? Should it not decode 64*(nb of behavioral variable) values, one for each time-step, since over 2.13s the variable can really change a lot (for example, for the head-direction speed of a mouse, an order of magnitude is 40deg/s).

Thank you for your help!

CYHSM commented 4 years ago

Thanks, hope you find it useful! You are correct, during the temporal window (64 time points) the behavioural variable might change a lot, that's why we use shared weights across the temporal dimension for the last layers (see the figure in the Readme or the code in architecture.py). This allows us to predict 4 timesteps (which is determined by how much we downsample the time dimension in the convolutional layers), with each time point corresponding roughly to 0.5s.

However, during model inference we can use a step size of one for predicting the behaviour, giving us a resolution of 30Hz (given that the original sampling rate=30000 and downsampling factor=1000)

PierreOrhan commented 4 years ago

Oh thank you! I thought the last 4 blocs of the picture were representing 4 output variables. Does it mean that, when testing, you are then sliding the (frequency-time-tetrode channel) input matrix over the time steps 1 by 1, then running the model and considering the prediction only for the first output? Thank you for your help,

P.s: in the paper there is a small error for the last figure, the column are misaligned: this is a bit disturbing, but I guess it will be changed at publication time, won't it?

CYHSM commented 4 years ago

Sorry for the confusion regarding the figure, the 4 blocks are actually timesteps. Yes exactly, the training is done with a stepsize of one, so for example the first training input might be 0:64, second 1:65, third 2:66, and so on (in practice I chose the input blocks randomly). The prediction however is for the last timestep (as this is the timestep which sees only the past), so with a 2.13s input I would predict the behaviour at the end of the 2.13 seconds.

Thanks for spotting this, I just fixed it in the manuscript!