test_custom.py always return 0

[iPsych]

test_custom.py seems generate random numpy array of (1,1,30000). However, it alway return predicted : tensor([0], device='cuda:0').

I modified the code a bit and run loop, but it's always 0. The exact command I used is

python test_custom.py --coinfig configs/SleePyCo-Transformer_SL-10_numScales-3_MASS_frezefinetune.json --fold 20 --gpu 0

Also the downloaded checkpoint name does not match with config file. I had to change it to SleePyCo-Transformer_SL-10_numscales-3_MASS_freezefinetune from SleePyCo-Transformer_SL-10_numscales-3_MASS-2018_freezefinetune

[SeongjuLee]

Hello, Thank you for having interests about our work. We fix the typo in download_checkpoints.py Also, we add missing lines in test_custom.py not to produce always zero.

[iPsych]

@SeongjuLee Thanks for the update. However, still always receive zero with intensive replication. Below is the code I run.

python test_custom.py --config configs/SleePyCo-Transformer_SL-10_numScales-3_MASS_freezefinetune.json --fold 20 --gpu 0

The return is always Predicted: tensor([0], device='cuda:0')

[SeongjuLee]

Did you take an input with your own data? If the model outputs constantly, your data might be pre-processed with different method compared to MASS data. If you let me know the characteristics of your data, it's going to be help for troubleshooting.

[iPsych]

@SeongjuLee Actually, I repeatedly run the test_custom.py, and always getting the same result as below. Predicted: tensor([0], device='cuda:0')

For custom data part, I loaded numpy array instead of input_data = np.random.rand(1, 1, 30000)

same 30000 samples of EEG data, resampled to 1000Hz.

Also always got Predicted: tensor([0], device='cuda:0') even I loaded different part of the whole EEG data recursively.

[SeongjuLee]

Our model takes 10 epochs of EEG data, sampled with 100 Hz (total 300s). While you sampled EEG with 1000 Hz, is it typo? The unit of EEG is the μV. Also, the EEG sample should be preprocessed by low pass filter with 0.30 Hz, high pass filter with 100 Hz.

[iPsych]

@SeongjuLee Thanks a lot. 1000Hz was my typo. The signal should be resampled with 100Hz, right? Finally the custom input data shows some result. Is there any 'normalizing' step already exist in the model, or I'd better to fit the amplitude range of the signal to the trained data?

[SeongjuLee]

Yes, we resampled with 100 Hz. There is no handcrafted normalization in the model. However, the learning-based normalization is included in the model such as BatchNorm in CNNs and LayerNorm in Transformer Layers. As i mentioned in the previous response, the only thing you check about the amplitude is to set the EEG scale in μV unit.

[Brandoncheaa]

@iPsych Do you think you could provide me with your data? I'm thinking my data could be the issue, I'm having similar issues where I am getting tensor[0].

[iPsych]

@Brandoncheaa Sadly, It's not lab data but hospital data, so I can't provide the data. It would be helpful to download several SHHS data used in train, and open it with same EDF tool (like pyedflib), and compare with your own data. Possibly, the amplitude range (mV, microVolt) might a issue.

[Brandoncheaa]

@iPsych Oh dang, I will try that. I have tried taking batches of the sleep-edf data and inputting it into test_custom.py however I keep getting the same results. Do you think you could possibly compare my data with yours and see if you can possibly point our any noticeable differences?

[iPsych]

@Brandoncheaa sure. load the data somewhere and put the link here. :) Don't forget to anonymize the edf file.