Compute accuracy of Tf-2 on SD wrt training size

[nipunbatra]

Yiling, see my email titled: Implementing Error v/s Train set size

also, see the notebook here: https://github.com/nipunbatra/scalable-nilm/blob/master/aaai/normal-learning.ipynb

Can you please change the notebook to implement the graph as per the email. I think by the time Hongning will reply, I will be asleep.

We won't analyse this results till the time I do data cleaning. However, it would be great to have the implementation ready by then.

[yilingjia]

No problem, I will do this immediately. One question about this, what do you mean by implementing the graph as “per the email”?

---

Yiling Jia Ph.D. Student Department of Computer Science University of Virginia

On Jun 14, 2017, at 11:06 AM, Nipun Batra notifications@github.com wrote:

Yiling, see my email titled: Implementing Error v/s Train set size

also, see the notebook here: https://github.com/nipunbatra/scalable-nilm/blob/master/aaai/normal-learning.ipynb https://github.com/nipunbatra/scalable-nilm/blob/master/aaai/normal-learning.ipynb Can you please change the notebook to implement the graph as per the email. I think by the time Hongning will reply, I will be asleep.

We won't analyse this results till the time I do data cleaning. However, it would be great to have the implementation ready by then.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/nipunbatra/scalable-nilm/issues/10, or mute the thread https://github.com/notifications/unsubscribe-auth/AVl-2mad9pjB2pLDvW7MhEqPPWnHOrp4ks5sD_blgaJpZM4N6BYi.

[nipunbatra]

I mean make a graph where we have on X-axis % of training homes (10, 20, ...100) and on Y-axis, we have the error rates for different appliances.

On Wed, Jun 14, 2017 at 8:40 PM, Yiling notifications@github.com wrote:

No problem, I will do this immediately. One question about this, what do you mean by implementing the graph as “per the email”?

---

Yiling Jia Ph.D. Student Department of Computer Science University of Virginia

On Jun 14, 2017, at 11:06 AM, Nipun Batra notifications@github.com wrote:

Yiling, see my email titled: Implementing Error v/s Train set size

also, see the notebook here: https://github.com/nipunbatra/ scalable-nilm/blob/master/aaai/normal-learning.ipynb https://github.com/ nipunbatra/scalable-nilm/blob/master/aaai/normal-learning.ipynb Can you please change the notebook to implement the graph as per the email. I think by the time Hongning will reply, I will be asleep.

We won't analyse this results till the time I do data cleaning. However, it would be great to have the implementation ready by then.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/ nipunbatra/scalable-nilm/issues/10, or mute the thread < https://github.com/notifications/unsubscribe-auth/AVl- 2mad9pjB2pLDvW7MhEqPPWnHOrp4ks5sD_blgaJpZM4N6BYi>.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/nipunbatra/scalable-nilm/issues/10#issuecomment-308462243, or mute the thread https://github.com/notifications/unsubscribe-auth/AADuOS8vihD0PBaXkwmSj4PvVQAjzkwoks5sD_fogaJpZM4N6BYi .

[yilingjia]

Do you mean this one?

[nipunbatra]

Yes.

On Wed, Jun 14, 2017 at 11:17 PM, Yiling notifications@github.com wrote:

[image: index] https://user-images.githubusercontent.com/22642394/27146626-f3981ae0-5107-11e7-9e0e-d9fbd0835ea6.png

Do you mean this one?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/nipunbatra/scalable-nilm/issues/10#issuecomment-308507269, or mute the thread https://github.com/notifications/unsubscribe-auth/AADuOb0YeQctWpeCmMIB1TqOkENc6yqJks5sEBykgaJpZM4N6BYi .

[yilingjia]

I use the cleaned data to run the experiments on SanDiego(without transfer learning). Just from the image:

1. the error seems become smaller for some appliances
2. the trend becomes flat, which means the data cleaning make the data more stable.

[nipunbatra]

Thank you.

Can you plot different appliances on separate subplots? Also, why does train percentage stop at 50? Also, can you add markets to points in the plot? Also, please share the code.

On 16 Jun 2017 01:53, "Yiling" notifications@github.com wrote:

I use the cleaned data to run the experiments on SanDiego. [image: unknown] https://user-images.githubusercontent.com/22642394/27200151-ba83041a-51e6-11e7-9265-5ebd7973b3ce.png Just from the image:

1. the error seems become smaller for some appliances
2. the trend becomes flat, which means the data cleaning make the data more stable.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/nipunbatra/scalable-nilm/issues/10#issuecomment-308855456, or mute the thread https://github.com/notifications/unsubscribe-auth/AADuOc_DLDiGKiI3fQTuhVX1w3n-QZ1Hks5sEZKkgaJpZM4N6BYi .

[yilingjia]

Hnn..I will do this immediately. For the train percentage, I just use the code from you in "normal-learning", and I will modify it and push it later.

[yilingjia]

1. Plot different appliances on separate subplots
2. plot appliances as a whole

And I have pushed the updated version of normal-learning.ipynb file:)

[nipunbatra]

Hi Yiling, Thanks for the graphs! I think there was some confusion. We wanted to implement as per the email I mentioned. Quoting here:

We do a KFold cross-validation. For example, let us take K=10. The 10th fold will serve as the test set. The remaining 9 folds will serve as the maximum train set. Of this maximum train set, we will obtain different subsets of train sets (say first 10% sample, first 20%,....100% samples). We will repeat this procedure rotating the Kth fold (as is usual practice in cross validation). This way, for each home, we'll have error rates for training on 10%, 20%,...100% of the train set.

I implemented this and pushed to: normal learning notebook. Let me know if you spot a bug!

[yilingjia]

Oh I see. So we only have one testing set, and the training is on different proportions of remaining data set. Is my understanding correct?

[nipunbatra]

We actually test on the entire dataset. Let me give an example. Say we have 20 homes.

[ 1, 2, ....20]

We do a 10-fold CV. For first fold, we have test set = [19, 20] and max train set as: [1, ...18] Now, we choose different prop. of this max train set. Say, when we choose 10%, we get [1, 2] When we choose 20%, we get [1, 2, 3, 4] and so on..

We then repeat the procedure for other folds as well. So, in next fold, we will test on [17, 18] and max train will be: [1, 2,...16, 19, 20]. Of this set we will choose different train proportions.

On Fri, Jun 16, 2017 at 8:32 PM, Yiling notifications@github.com wrote:

Oh I see. So we only have one testing set, and the training is on different proportion of remaining data set. Is my understanding correct?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/nipunbatra/scalable-nilm/issues/10#issuecomment-309050247, or mute the thread https://github.com/notifications/unsubscribe-auth/AADuOUJJ3RxyZpNAQSBN-8ucTwYRZZhYks5sEpkbgaJpZM4N6BYi .