search

zhhlee / InterFusion

KDD 2021: Multivariate Time Series Anomaly Detection and Interpretation using Hierarchical Inter-Metric and Temporal Embedding
MIT License
200 stars 46 forks source link

Error in running SWaT dataset #2

Closed yongmei5 closed 3 years ago

yongmei5 commented 3 years ago

Thank you so much for providing the code this great work!

Currently, I am able to run the command python algorithm/stack_train.py --dataset=omi-1, but I have some issues in running the command python algorithm/stack_train.py --dataset=SWaT --train.train_start=21600 --train.valid_portion=0.1 --model.window_length=30 '--model.output_shape=[15, 15, 30]' .

I have gained the access to SWAT dataset, and I saw that in the physical folder in the 2015 version, there exists "SWaT_Dataset_Attack_v0.xlsx" and "SWaT_Dataset_Normal_v1.xlsx" rather than "SWaT_Dataset_Attack_v0.csv" and "SWaT_Dataset_Normal_v0.csv" files. So I just used xlsx2csv SWaT_Dataset_Attack_v0.xlsx > SWaT_Dataset_Attack_v0.csv. The same applied for the SWaT_Dataset_Normal_v0.csv

After using the function explib/raw_data_converter to convert the respective csv files to pkl files, I put the corresponding SWaT_test_label.pkl, SWaT_test.pkl, and SWaT_train.pkl in the data/processed folder.

After running the command python algorithm/stack_train.py --dataset=SWaT --train.train_start=21600 --train.valid_portion=0.1 --model.window_length=30 '--model.output_shape=[15, 15, 30]' I am able to load the SWaT dataset successfully. However, I am keeping getting this error:

 envs/interFusion/lib/python3.6/site-packages/ten
sorflow/python/framework/common_shapes.py", line 596, in broadcast_shape
    % (shape_x, shape_y))
ValueError: Incompatible shapes for broadcasting: (?, 8, 51) and (13, 51) 

ValueError: The given argument should be able to broadcast to match batch_shape + value_shape of the distribution. ((?, 8, 51) vs. (13, 51) + ())

Can you please give me some help? Is there anything that went wrong with the steps I described above? Many thanks!!!

zhhlee commented 3 years ago

@yongmei5 Oops, I omitted a parameter '--model.z2_dim=8' for running SWaT and WADI in the readme file (since the default model configuration is from InterFusion.py, where model.z2_dim=13). Now I have updated the readme file and fixed it.

yongmei5 commented 3 years ago

@zhhlee, Thanks so much. The code was able to run after adding the flags.