search

HelmchenLabSoftware / Cascade

Calibrated inference of spiking from calcium ΔF/F data using deep networks
GNU General Public License v3.0
123 stars 34 forks source link

"Global_EXC_15Hz_smoothing50ms" model mislabeled? #45

Closed kimtonyhyun closed 2 years ago

kimtonyhyun commented 2 years ago

Hi Peter,

I was looking at recent updates to this repo and noted that the Global_EXC_15Hz_smoothing50ms model was added recently. I was curious about this model (previously, I was using Global_EXC_15Hz_smoothing100ms) so I attempted to download and run the 50ms model.

When attempting to do so (cascade.predict(...)), I found the following text output:

The selected model was trained on 18 datasets, with 5 ensembles for each noise level, at a sampling rate of 3Hz, with a resampled ground truth that was smoothed with a Gaussian kernel of a standard deviation of 50 milliseconds. 

Loaded model was trained at frame rate 3 Hz
...

(and I ran into a Python error that I'm not sure is related).

Could it be that the Global_EXC_15Hz_smoothing50ms model was mislabeled, i.e. either it's actually a 3 Hz model, or it's in fact a 15 Hz model but with an incorrect descriptor?

-Tony

PTRRupprecht commented 2 years ago

Hi Tony,

Thanks for noting this issue! I'll look into this problem during the next days and will keep you updated!

PTRRupprecht commented 2 years ago

Hi again,

you are absolutely right, the Global_EXC_15Hz_smoothing50ms model is indeed a 3 Hz model. I will retrain the affected models (Global_EXC_15Hz_smoothing50ms and Global_EXC_15Hz_smoothing50ms_causalkernel) and will upload them as soon as they are trained.

I also checked all other pretrained models for any inconsistencies, and I found that one model (Global_EXC_30Hz_smoothing100ms_causal) contained models for only lower noise level ground truth, and I found another model (Global_EXC_1Hz_smoothing500ms) which was trained with a smoothing factor of 1000 ms instead of 500 ms. I will retrain these models and update them soon as well.

Thanks a lot for spotting this bug!

If you still have to deal with this Python error you mentioned, it would be best to open a new issue about it and mention whether it occurred when using the Colab version or your locally installed version of Cascade.

Peter

PTRRupprecht commented 2 years ago

Hi Tony @kimtonyhyun.

The problem should be fixed now - I've retrained the models mentioned above and uploaded them to the database of pretrained models. If you want to use them, please delete the previous specific pretrained model and download again from the server.

Let me know if you have any remaining problems!

Peter