search

saharhzm / CrossModalSleepTransformer

The Cross-modal SleepTransformer is a deep learning model designed for sleep stage classification based on multimodal physiological signals. It utilizes the transformer architecture, and cross-attention mechanism to effectively process and integrate information from multiple input modalities, including EEG, EOG, EMG, ECG, and respiratory signals.
MIT License
2 stars 1 forks source link

Where to use handcraft-EEG in the model #2

Open shqing opened 2 months ago

shqing commented 2 months ago

Hi， I see the following code:
handcraft_EEG = np.load(data_dir+'X_EEG.npy',allow_pickle=True) handcraft_EMG = np.load(data_dir+'X_EMG.npy',allow_pickle=True) handcraft_ECG = np.load(data_dir+'X_ECG.npy',allow_pickle=True)

However, these features， such as handcraft_EEG , handcraft_ECG, are not used in the code.

saharhzm commented 2 months ago

Hi, I used it in line 126 of train_model to integrate it with raw data features

features2 = layers.Concatenate(axis=1)([EEG_HC, EMG_HC, ECG_HC, EOG_HC, NEWAIR_HC, THOR_HC, ABDO_HC])

transformer_inputs2 = layers.Dense(448, activation="relu")(features2) transformer_inputs2 = layers.Reshape((7, 64), input_shape=transformer_inputs2.shape)(transformer_inputs2)

cross_modal_transformer = CrossModalTransformer(d_model= 512, num_heads= 4, head_size=64, dff=64) cross_modal_transformer_out = cross_modal_transformer(transformer_inputs1, transformer_inputs2)