search

MingjunZhong / transferNILM

Transfer Learning for Non-Intrusive Load Monitoring
114 stars 44 forks source link

Problems with the training and with the plot of the results #14

Open riccardosalvucci99 opened 1 year ago

riccardosalvucci99 commented 1 year ago

Hi, I'm using the code for the transfer nilm. When I run the seq2point_train.py with the dataset refit for the washing machine I obtain this result:

Epoch 5 of 5 -------------------------- 2023-10-16 21:50:53,666 [INFO ] loss_ep: 0.021295 2023-10-16 21:50:53,666 [INFO ] Epoch 5 of 5 took 18.028538s 2023-10-16 21:50:53,666 [INFO ] Validation... 2023-10-16 21:50:58,571 [INFO ] train loss/n_batch_train: 0.020180 2023-10-16 21:50:58,571 [INFO ] train loss: 0.201802, n_batch_train: 10 2023-10-16 21:51:03,492 [INFO ] val loss: 0.000374 2023-10-16 21:51:03,492 [INFO ] Evaluate earlystopping parameters... 2023-10-16 21:51:03,492 [INFO ] Current valid loss was 0.000374, acc was 0.000000, train loss was 0.020180, acc was 0.000000 at epoch 5. 2023-10-16 21:51:04,121 [INFO ] Best valid loss was 0.000374 and acc 0.000000 at epoch 5. 2023-10-16 21:51:04,122 [INFO ] Total training time: 141.299270s

Why is the accuracy value 0.0? Am I doing something wrong?

Then I've a problem with the plot of the results when I run the seq2point_test.py. I've no problems with the test of the network in fact I obtain the values of the parameters such as the NDE, SAE, MAE, Energy per Day but, at the end of the test, I don't see the figure with the results.

Thank you very much.

Riccardo

MingjunZhong commented 1 year ago

The acc was not computed because we only care about NDE, SAE, MAE, etc. For plotting the results, have you set the args.plot_results=True?

riccardosalvucci99 commented 1 year ago

Thank you very much for the answer. Okay, I understand. So the fact that the accuracy is 0, it doesn't affect then the performance of the network right?

Yes I set arg.plot_results=True and I'm running the seq2point_test.py with Visual Studio Code

MingjunZhong commented 1 year ago

True - accuracy 0.00 won't affect anything.

riccardosalvucci99 commented 1 year ago

Hello.

This morning I managed to plot the results. Nevertheless, when I run seq2point_test.py with the network trained on the washingmachine of the dataset REFIT, I obtain this result: why is the prediction line 0? These are the values of the parameters obtained with the test: 2023-10-19 11:39:10,676 [INFO ] aggregate_mean: 522 2023-10-19 11:39:10,676 [INFO ] aggregate_std: 814 2023-10-19 11:39:10,676 [INFO ] appliance_mean: 400 2023-10-19 11:39:10,676 [INFO ] appliance_std: 700 2023-10-19 11:39:10,814 [INFO ] tp=6117871.0 2023-10-19 11:39:10,814 [INFO ] fn=0.0 2023-10-19 11:39:10,814 [INFO ] 1.0 2023-10-19 11:39:10,909 [INFO ] tp=6117871.0 2023-10-19 11:39:10,909 [INFO ] fp=0.0 2023-10-19 11:39:10,909 [INFO ] 1.0 2023-10-19 11:39:10,909 [INFO ] F1:1.0 2023-10-19 11:39:10,942 [INFO ] NDE:1.0005341616229328 2023-10-19 11:39:11,505 [INFO ]
MAE: 32.52002318104955 -std: 243.8659484305266 -min: 3.051757818184342e-05 -max: 3972.002 -q1: 0.0009999999999763531 -median: 0.0009999999999763531 -q2: 5.313140869140625 2023-10-19 11:39:11,538 [INFO ] SAE: 0.88866547736957 2023-10-19 11:39:11,563 [INFO ] Energy per Day: 695.1830117782172 plot

MingjunZhong commented 1 year ago

That looks like the model was not learning. Does it have the same problem for other appliances? You also need to be careful to make sure the prediction was transformed back, because when training, the data was normalised.

riccardosalvucci99 commented 1 year ago

I've tried to test the fridge (REFIT) with the network trained on the fridge (REFIT), but I obtain the same thing. The trend of the ground truth of the fridge obviously is different from that one of the washingmachine (same thing for the aggregate). I don't understand which is the problem, the code is exactly like yours, I've imported it.