about feature size

[holydick99]

In config.yaml file there is a "feature_size" I don't know this depend on wlich feature of data. My data shape is (samples, seq_length) = (10000,178), Looking for your reply

[Y-debug-sys]

Hi, holydick99. In fact, your size is (10000,178, 1), which means you need first expand your tensor at the last dim, then set feature_size=1 in the config.yaml. Thanks!

[holydick99]

Hi, holydick99. In fact, your size is (10000,178, 1), which means you need first expand your tensor at the last dim, then set feature_size=1 in the config.yaml. Thanks!

Thanks for your answer!

[holydick99]

Hi, holydick99. In fact, your size is (10000,178, 1), which means you need first expand your tensor at the last dim, then set feature_size=1 in the config.yaml. Thanks!

where should this step be added, at beginning my data shape is (1w, 1, 178) but ERROR about this: /Diffusion-TS-test/Utils/Data_utils/real_datasets.py:218, in EpilepsyDataset.read_data(filepath, name) 215 """Reads a single .csv 216 """ 217 data = io.loadmat(filepath + '/pseudo_labels_epilepsy.mat')['Epilepsy'] 219 scaler = MinMaxScaler() 220 scaler = scaler.fit(data) except dim <= 2

So, you mean after data scaler then add dim?

[Y-debug-sys]

Hi, please reshape your data before Minmax Normalization. The scaler only support a shape like (x, y) or (x, ). The code is:

data = io.loadmat(filepath + '/pseudo_labels_epilepsy.mat')['Epilepsy']
scaler = MinMaxScaler()
scaler = scaler.fit(data.reshape(-1, 1))

[holydick99]

Hi, please expand your data after Minmax Normalization. The scaler only support a shape like (x, y) or (x, ).

hi, data and scaler both should be expand？ or just data

[Y-debug-sys]

Sorry, I have corrected the answer.

[Y-debug-sys]

Then you can normalize the ground truth by data_norm = scaler.transform(data.reshape(-1,1)).reshape(data.shape) and obtain your unnormalized generation by gen = scaler.inverse_transform(gen.reshape(-1,1)).reshape(gen.shape)

[holydick99]

Then you can normalize the ground truth by data_norm = scaler.transform(data.reshape(-1,1)).reshape(data.shape) and obtain your unnormalized generation by gen = scaler.inverse_transform(gen.reshape(-1,1)).reshape(gen.shape)

these two lines code are little confused me. First, I'm curious about how to set feature-size. And my data in dataset is (samples, 1 ,seq_length), before I scaler it I use sequeeze to fit. What's more, only if I set the feature-size = seq_length the model can be trained. So there is any problem with my step? Or you can corrected my step! thank you.

[Y-debug-sys]

I think you should refer to mujoco_dataset.py and sine_dataset.py more. The only difference is the part of loading data; the other two directly synthesis data with dimensions (num_samples, seq_length, feature_size).

[holydick99]

I think you should refer to mujoco_dataset.py and sine_dataset.py more. The only difference is the part of loading data; the other two directly synthesis data with dimensions (num_samples, seq_length, feature_size).

OK, thanks for your reply!