search

gist-ailab / SleePyCo

This is an official implementation for "SleePyCo: Automatic Sleep Scoring with Feature Pyramid and Contrastive Learning".
MIT License
42 stars 12 forks source link

Formatting for custom data #10

Open Brandoncheaa opened 4 months ago

Brandoncheaa commented 4 months ago

Previously you mentioned the highpass cutoff being at 100Hz and the lowpass being at 0.3Hz however when I run those values I get errors:

"Traceback (most recent call last): File "train_model.py", line 158, in main() File "train_model.py", line 120, in main generated_data_filtered = apply_filter(generated_data_resampled, low_cutoff, high_cutoff, fs) File "/cluster/academic/projects/ASCS/env/GAN-Sleep/utils.py", line 219, in apply_filter b_high, a_high = butter_highpass(high_cutoff, fs, order=order) File "/cluster/academic/projects/ASCS/env/GAN-Sleep/utils.py", line 211, in butter_highpass raise ValueError(f"Invalid highpass filter cutoff frequency: {cutoff} Hz (normalized: {normal_cutoff})") ValueError: Invalid highpass filter cutoff frequency: 100 Hz (normalized: 2.0)"

Brandoncheaa commented 4 months ago

My concern seems to lie with the high-pass cutoff frequency being at 100 Hz it is beyond the Nyquist frequency for a sampling rate of 100 Hz, making the normalized frequency greater than 1 which I think it causing my error.

SeongjuLee commented 4 months ago

I also agree with that the reason why you get error is due to Nyquist frequency. If you have signals with sampled larger than 200 Hz, preprocess your signal with the following process: applying high-pass and low-pass filter -> resample with 100 Hz If the sampling rate of signal you have is smaller than 200 Hz, apply 30 Hz filter as preprocessed in Sleep-EDF dataset.

Brandoncheaa commented 4 months ago

I have taken a more simple approach, generating data not using GAN. Here is my dataset dimensions at (1,1,30000) sampled at 100Hz for 300s. I have provided the datset. The issue I have is similar to a previous user the Tensor[0], is the test_custom.py supposed to provide some sort of confusion matrix for how the custom data tests against the train data? eeg_data_reshaped.csv

I even took a part of the sleep-edf-2013 dataset and cut them into batches with the same dimensions that are specified, however it is still showing tensor([0], device='cuda:0').