Help on recreating dict per house

[aleonnet]

Hi @ZhangRaymond, I really tried to recreate this dict, but I could not handle nilmtk/REDD adequately I think :( I'm sorry. Please help me. I'd really like to try your code out.

Thank you. import pickle from nilmtk import DataSet, TimeFrame import pandas as pd

datafolder = '/Users/alessandro/Documents/data/'

redd = DataSet(datafolder + 'redd.h5') metadata = dict(redd.metadata)

deviceName = set() appName = set() mains={} appliances={} for i in range(1,7): mains[i] = {}

for house in range(1,7): print('house_',house) elec = redd.buildings[house].elec appliancelist = elec.appliances mainsvalues = elec.mains().power_series_all_data() mains[house]=mainsvalues for app in appliancelist: label = app.label('unknown') print(' '*3,label) appName.add(label) appliances[house][label] = elec[label].power_series(actype='active') deviceName = list(appName) with open('house{}.pickle'.format(house), 'wb') as file: pickle.dump([deviceName], file) print('End')

[ZhangRaymond]

Hi, Aleonnet,

I am too busy for my final thesis these days and I will help you to recode this when I am free. Forgive me for my busyness.

Regards, Raymond

---

发件人: aleonnet notifications@github.com 发送时间: Saturday, April 20, 2019 11:38:36 AM 收件人: ZhangRaymond/Neural-NILM 抄送: Raymond, Zhang Yuanmeng; Mention 主题: [ZhangRaymond/Neural-NILM] Help on recreating dict per house (#2)

Hi @ZhangRaymondhttps://github.com/ZhangRaymond, I really tried to recreate this dict, but I could not handle nilmtk/REDD adequately I think :( I'm sorry. Please help me. I'd really like to try your code out.

[image]https://user-images.githubusercontent.com/20618327/56451834-14b5f980-6304-11e9-9fc2-5ac7c6e2313f.png

Thank you. import pickle from nilmtk import DataSet, TimeFrame import pandas as pd

datafolder = '/Users/alessandro/Documents/data/'

redd = DataSet(datafolder + 'redd.h5') metadata = dict(redd.metadata)

deviceName = set() appName = set() mains={} appliances={} for i in range(1,7): mains[i] = {}

for house in range(1,7): print('house_',house) elec = redd.buildings[house].elec appliancelist = elec.appliances mainsvalues = elec.mains().power_series_all_data() mains[house]=mainsvalues for app in appliancelist: label = app.label('unknown') print(' '*3,label) appName.add(label) appliances[house][label] = elec[label].power_series(actype='active') deviceName = list(appName) with open('house{}.pickle'.format(house), 'wb') as file: pickle.dump([deviceName], file) print('End')

― You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/ZhangRaymond/Neural-NILM/issues/2, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AD23A4QEUICH7QMPLOWFU4TPRKF3ZANCNFSM4HHI3CDQ.

[aleonnet]

Hi Raymond, Sorry to trouble you, I perfectly undeserving. I’d really like to follow up your results on the article and code.

Thank you. Regards, Alessandro

On Sat, 27 Apr 2019 at 02:48 Raymond Zhang notifications@github.com wrote:

Hi, Aleonnet,

I am too busy for my final thesis these days and I will help you to recode this when I am free. Forgive me for my busyness.

Regards, Raymond

---

发件人: aleonnet notifications@github.com 发送时间: Saturday, April 20, 2019 11:38:36 AM 收件人: ZhangRaymond/Neural-NILM 抄送: Raymond, Zhang Yuanmeng; Mention 主题: [ZhangRaymond/Neural-NILM] Help on recreating dict per house (#2)

Hi @ZhangRaymondhttps://github.com/ZhangRaymond, I really tried to recreate this dict, but I could not handle nilmtk/REDD adequately I think :( I'm sorry. Please help me. I'd really like to try your code out.

[image]< https://user-images.githubusercontent.com/20618327/56451834-14b5f980-6304-11e9-9fc2-5ac7c6e2313f.png>

Thank you. import pickle from nilmtk import DataSet, TimeFrame import pandas as pd

datafolder = '/Users/alessandro/Documents/data/'

redd = DataSet(datafolder + 'redd.h5') metadata = dict(redd.metadata)

deviceName = set() appName = set() mains={} appliances={} for i in range(1,7): mains[i] = {}

for house in range(1,7): print('house_',house) elec = redd.buildings[house].elec appliancelist = elec.appliances mainsvalues = elec.mains().power_series_all_data() mains[house]=mainsvalues for app in appliancelist: label = app.label('unknown') print(' '*3,label) appName.add(label) appliances[house][label] = elec[label].power_series(actype='active') deviceName = list(appName) with open('house{}.pickle'.format(house), 'wb') as file: pickle.dump([deviceName], file) print('End')

― You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub< https://github.com/ZhangRaymond/Neural-NILM/issues/2>, or mute the thread< https://github.com/notifications/unsubscribe-auth/AD23A4QEUICH7QMPLOWFU4TPRKF3ZANCNFSM4HHI3CDQ>.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ZhangRaymond/Neural-NILM/issues/2#issuecomment-487257689, or mute the thread https://github.com/notifications/unsubscribe-auth/AE5JYV7PDJ5LD766H37MGILPSPSJVANCNFSM4HHI3CDQ .

[aleonnet]

Hi Raymond, I was able to recreate the house pickle structure with the following code:

[image: image.png]

[image: image.png]

But a few other issues came up: 1- The 'deviceName' names refer to the nilmtk standard types. I've not found any documented method/metadata to get 'original_name' appliances from REDD. 2- Only house_2, shows the right structure according to the picture you sent me. For example, check house1 below. Some appliances have more samples than others. How to solve that? [image: image.png] house 1 unknown (745878, 1) unknown (745878, 1) fridge (745878, 1) sockets (745878, 1) electric oven Loading data for meter ElecMeterID(instance=4, building=1, dataset='REDD') Done loading data all meters for this chunk. (1044688, 1) 3- The pickle file struture(keys) seems to be correct, but get_house_raw_data in lib.py is still raising an AttributeError: 'list' object has no attribute 'deviceName' and it has it as you can see above.

[image: image.png]

Would you help me pls? It took me a lot of time to get to this point.

Thank you. Alessandro

On Sat, Apr 27, 2019 at 2:48 AM Raymond Zhang notifications@github.com wrote:

Hi, Aleonnet,

I am too busy for my final thesis these days and I will help you to recode this when I am free. Forgive me for my busyness.

Regards, Raymond

---

发件人: aleonnet notifications@github.com 发送时间: Saturday, April 20, 2019 11:38:36 AM 收件人: ZhangRaymond/Neural-NILM 抄送: Raymond, Zhang Yuanmeng; Mention 主题: [ZhangRaymond/Neural-NILM] Help on recreating dict per house (#2)

Hi @ZhangRaymondhttps://github.com/ZhangRaymond, I really tried to recreate this dict, but I could not handle nilmtk/REDD adequately I think :( I'm sorry. Please help me. I'd really like to try your code out.

[image]< https://user-images.githubusercontent.com/20618327/56451834-14b5f980-6304-11e9-9fc2-5ac7c6e2313f.png>

Thank you. import pickle from nilmtk import DataSet, TimeFrame import pandas as pd

datafolder = '/Users/alessandro/Documents/data/'

redd = DataSet(datafolder + 'redd.h5') metadata = dict(redd.metadata)

deviceName = set() appName = set() mains={} appliances={} for i in range(1,7): mains[i] = {}

for house in range(1,7): print('house_',house) elec = redd.buildings[house].elec appliancelist = elec.appliances mainsvalues = elec.mains().power_series_all_data() mains[house]=mainsvalues for app in appliancelist: label = app.label('unknown') print(' '*3,label) appName.add(label) appliances[house][label] = elec[label].power_series(actype='active') deviceName = list(appName) with open('house{}.pickle'.format(house), 'wb') as file: pickle.dump([deviceName], file) print('End')

― You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub< https://github.com/ZhangRaymond/Neural-NILM/issues/2>, or mute the thread< https://github.com/notifications/unsubscribe-auth/AD23A4QEUICH7QMPLOWFU4TPRKF3ZANCNFSM4HHI3CDQ>.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ZhangRaymond/Neural-NILM/issues/2#issuecomment-487257689, or mute the thread https://github.com/notifications/unsubscribe-auth/AE5JYV7PDJ5LD766H37MGILPSPSJVANCNFSM4HHI3CDQ .

[aleonnet]

Hi

On Sun, 2 Jun 2019 at 17:03 Alessandro Barbosa barbosa.alessandro@gmail.com wrote:

Hi Raymond, I was able to recreate the house pickle structure with the following code:

[image: image.png]

[image: image.png]

But a few other issues came up: 1- The 'deviceName' names refer to the nilmtk standard types. I've not found any documented method/metadata to get 'original_name' appliances from REDD. 2- Only house_2, shows the right structure according to the picture you sent me. For example, check house1 below. Some appliances have more samples than others. How to solve that? [image: image.png] house 1 unknown (745878, 1) unknown (745878, 1) fridge (745878, 1) sockets (745878, 1) electric oven Loading data for meter ElecMeterID(instance=4, building=1, dataset='REDD') Done loading data all meters for this chunk. (1044688, 1) 3- The pickle file struture(keys) seems to be correct, but get_house_raw_data in lib.py is still raising an AttributeError: 'list' object has no attribute 'deviceName' and it has it as you can see above.

[image: image.png]

Would you help me pls? It took me a lot of time to get to this point.

Thank you. Alessandro

On Sat, Apr 27, 2019 at 2:48 AM Raymond Zhang notifications@github.com wrote:

Hi, Aleonnet,

I am too busy for my final thesis these days and I will help you to recode this when I am free. Forgive me for my busyness.

Regards, Raymond

---

发件人: aleonnet notifications@github.com 发送时间: Saturday, April 20, 2019 11:38:36 AM 收件人: ZhangRaymond/Neural-NILM 抄送: Raymond, Zhang Yuanmeng; Mention 主题: [ZhangRaymond/Neural-NILM] Help on recreating dict per house (#2)

Hi @ZhangRaymondhttps://github.com/ZhangRaymond, I really tried to recreate this dict, but I could not handle nilmtk/REDD adequately I think :( I'm sorry. Please help me. I'd really like to try your code out.

[image]< https://user-images.githubusercontent.com/20618327/56451834-14b5f980-6304-11e9-9fc2-5ac7c6e2313f.png>

Thank you. import pickle from nilmtk import DataSet, TimeFrame import pandas as pd

datafolder = '/Users/alessandro/Documents/data/'

redd = DataSet(datafolder + 'redd.h5') metadata = dict(redd.metadata)

deviceName = set() appName = set() mains={} appliances={} for i in range(1,7): mains[i] = {}

for house in range(1,7): print('house_',house) elec = redd.buildings[house].elec appliancelist = elec.appliances mainsvalues = elec.mains().power_series_all_data() mains[house]=mainsvalues for app in appliancelist: label = app.label('unknown') print(' '*3,label) appName.add(label) appliances[house][label] = elec[label].power_series(actype='active') deviceName = list(appName) with open('house{}.pickle'.format(house), 'wb') as file: pickle.dump([deviceName], file) print('End')

― You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub< https://github.com/ZhangRaymond/Neural-NILM/issues/2>, or mute the thread< https://github.com/notifications/unsubscribe-auth/AD23A4QEUICH7QMPLOWFU4TPRKF3ZANCNFSM4HHI3CDQ>.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ZhangRaymond/Neural-NILM/issues/2#issuecomment-487257689, or mute the thread https://github.com/notifications/unsubscribe-auth/AE5JYV7PDJ5LD766H37MGILPSPSJVANCNFSM4HHI3CDQ .

[ZhangRaymond]

Hi On Sun, 2 Jun 2019 at 17:03 Alessandro Barbosa barbosa.alessandro@gmail.com wrote: … Hi Raymond, I was able to recreate the house pickle structure with the following code: [image: image.png] [image: image.png] But a few other issues came up: 1- The 'deviceName' names refer to the nilmtk standard types. I've not found any documented method/metadata to get 'original_name' appliances from REDD. 2- Only house_2, shows the right structure according to the picture you sent me. For example, check house1 below. Some appliances have more samples than others. How to solve that? [image: image.png] house 1 unknown (745878, 1) unknown (745878, 1) fridge (745878, 1) sockets (745878, 1) electric oven Loading data for meter ElecMeterID(instance=4, building=1, dataset='REDD') Done loading data all meters for this chunk. (1044688, 1) 3- The pickle file struture(keys) seems to be correct, but get_house_rawdata in lib.py is still raising an AttributeError: 'list' object has no attribute 'deviceName' and it has it as you can see above. [image: image.png] Would you help me pls? It took me a lot of time to get to this point. Thank you. Alessandro On Sat, Apr 27, 2019 at 2:48 AM Raymond Zhang @.> wrote: > Hi, Aleonnet, > > I am too busy for my final thesis these days and I will help you to > recode this when I am free. Forgive me for my busyness. > > Regards, > Raymond > > ____ > 发件人: aleonnet @.> > 发送时间: Saturday, April 20, 2019 11:38:36 AM > 收件人: ZhangRaymond/Neural-NILM > 抄送: Raymond, Zhang Yuanmeng; Mention > 主题: [ZhangRaymond/Neural-NILM] Help on recreating dict per house (#2) > > > Hi @ZhangRaymondhttps://github.com/ZhangRaymond, > I really tried to recreate this dict, but I could not handle nilmtk/REDD > adequately I think :( I'm sorry. > Please help me. I'd really like to try your code out. > > [image]< > https://user-images.githubusercontent.com/20618327/56451834-14b5f980-6304-11e9-9fc2-5ac7c6e2313f.png> > > > Thank you. > import pickle > from nilmtk import DataSet, TimeFrame > import pandas as pd > > datafolder = '/Users/alessandro/Documents/data/' > > redd = DataSet(datafolder + 'redd.h5') > metadata = dict(redd.metadata) > > deviceName = set() > appName = set() > mains={} > appliances={} > for i in range(1,7): > mains[i] = {} > > for house in range(1,7): > print('house',house) > elec = redd.buildings[house].elec > appliancelist = elec.appliances > mainsvalues = elec.mains().power_series_all_data() > mains[house]=mainsvalues > for app in appliancelist: > label = app.label('unknown') > print(' '*3,label) > appName.add(label) > appliances[house][label] = elec[label].power_series(actype='active') > deviceName = list(appName) > with open('house{}.pickle'.format(house), 'wb') as file: > pickle.dump([deviceName], file) > print('End') > > ― > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub< > #2>, or mute the > thread< > https://github.com/notifications/unsubscribe-auth/AD23A4QEUICH7QMPLOWFU4TPRKF3ZANCNFSM4HHI3CDQ>. > > > — > You are receiving this because you authored the thread. > Reply to this email directly, view it on GitHub > <#2 (comment)>, > or mute the thread > https://github.com/notifications/unsubscribe-auth/AE5JYV7PDJ5LD766H37MGILPSPSJVANCNFSM4HHI3CDQ > . >

Hi, Aleonnet,

Sorry for the late reply, I have just finished my thesis. I can't load your figures so I don't know what's wrong for you code, but I will rewrite the code for you soon.

Regards, Raymond

[ZhangRaymond]

Hi，@aleonnet，

I am currently using the extracted data, one file per house. It didn't uploaded to github before because these files are too large (341MB). Just now I am surprised to find that there is only 7MB after compression, so I uploaded the compressed package and committed just now, please unzip it and use it directly. I'll take the time to rewrite the generating code.

[aleonnet]

Hi Raymond, I’ve got the code working! I’ll send it to you. I’m surprised to learn that the training is taking so long about 31h per epoch. Is this correct?

Regards,

On Mon, 10 Jun 2019 at 12:29 Raymond Zhang notifications@github.com wrote:

Hi，@aleonnet https://github.com/aleonnet，

I am currently using the extracted data, one file per house. It didn't uploaded to github before because these files are too large (341MB). Just now I am surprised to find that there is only 7MB after compression, so I uploaded the compressed package and committed just now, please unzip it and use it directly. I'll take the time to rewrite the generating code.

—

You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/ZhangRaymond/Neural-NILM/issues/2?email_source=notifications&email_token=AE5JYV355GHSMYQGPXYACHDPZZXORA5CNFSM4HHI3CD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXKGUOA#issuecomment-500460088, or mute the thread https://github.com/notifications/unsubscribe-auth/AE5JYV5VBXAL2WLQT7HDTVDPZZXORANCNFSM4HHI3CDQ .

[ZhangRaymond]

Hi，@aleonnet， Congrats for your code! The training time depends on hardware and configurations of experiment especially the length of window size. I don't know what's wrong for your code or handware but I can share my training time: 20 ~ 120 mins/epoch on CPU (i7-7) or 1 ~ 20 mins/epoch on GPU (P40).

Good luck for you!

Regards, Raymond

[aleonnet]

Hi @ZhangRaymond Thank you! Have you published your results (thesis or article)? I'd like to check them out before moving forward (still 20min/epoch is plenty of power!) What about some unsupervised or semi supervised approach?

Regards,

PS: Check my contributions at https://github.com/aleonnet/GSP_energy_disaggregator

[hanyan0705]

In line 25 of Seq2point.py (from packages import *), the run shows ModuleNotFoundError: No module named 'packages'.What caused this?

[Mars0207]

Hi @ZhangRaymond, I'm sorry to trouble you, when I run the code 'main', I met a problem called 'get_modelPath() missing 1 required positional argument: 'dir_path'' , I can not find a file named 'model.h5'. May I have your help? Thank you, Ying Zhao

[WT19960917]

Hi Raymond, I’ve got the code working! I’ll send it to you. I’m surprised to learn that the training is taking so long about 31h per epoch. Is this correct? Regards, … On Mon, 10 Jun 2019 at 12:29 Raymond Zhang @.> wrote: @. <https://github.com/aleonnet>， I am currently using the extracted data, one file per house. It didn't uploaded to github before because these files are too large (341MB). Just now I am surprised to find that there is only 7MB after compression, so I uploaded the compressed package and committed just now, please unzip it and use it directly. I'll take the time to rewrite the generating code. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2?email_source=notifications&email_token=AE5JYV355GHSMYQGPXYACHDPZZXORA5CNFSM4HHI3CD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXKGUOA#issuecomment-500460088>, or mute the thread https://github.com/notifications/unsubscribe-auth/AE5JYV5VBXAL2WLQT7HDTVDPZZXORANCNFSM4HHI3CDQ .

hi，aleonnet, I want to know how you create the.pickle file for each house. Can you provide me with the code you wrote? Thank you. Regards, Wang Tao

[WT19960917]

Hi Raymond, I’ve got the code working! I’ll send it to you. I’m surprised to learn that the training is taking so long about 31h per epoch. Is this correct? Regards, … On Mon, 10 Jun 2019 at 12:29 Raymond Zhang @.**> wrote: @.** <https://github.com/aleonnet>， I am currently using the extracted data, one file per house. It didn't uploaded to github before because these files are too large (341MB). Just now I am surprised to find that there is only 7MB after compression, so I uploaded the compressed package and committed just now, please unzip it and use it directly. I'll take the time to rewrite the generating code. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2?email_source=notifications&email_token=AE5JYV355GHSMYQGPXYACHDPZZXORA5CNFSM4HHI3CD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXKGUOA#issuecomment-500460088>, or mute the thread https://github.com/notifications/unsubscribe-auth/AE5JYV5VBXAL2WLQT7HDTVDPZZXORANCNFSM4HHI3CDQ .

hi，aleonnet, I want to know how you create the.pickle file for each house. Can you provide me with the code you wrote? Thank you. Regards, Wang Tao

Hi, aleonnet，

I have found the code for get_house_data.py in your repository. I am new to Github and thank you very much for providing the code.

Regards, Wang Tao

[Diana1314]

Have you finished recreating this dict?

[DuoCaiXie]

Hi Raymond, I’ve got the code working! I’ll send it to you. I’m surprised to learn that the training is taking so long about 31h per epoch. Is this correct? Regards, … On Mon, 10 Jun 2019 at 12:29 Raymond Zhang @.> wrote: @. <https://github.com/aleonnet>， I am currently using the extracted data, one file per house. It didn't uploaded to github before because these files are too large (341MB). Just now I am surprised to find that there is only 7MB after compression, so I uploaded the compressed package and committed just now, please unzip it and use it directly. I'll take the time to rewrite the generating code. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2?email_source=notifications&email_token=AE5JYV355GHSMYQGPXYACHDPZZXORA5CNFSM4HHI3CD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXKGUOA#issuecomment-500460088>, or mute the thread https://github.com/notifications/unsubscribe-auth/AE5JYV5VBXAL2WLQT7HDTVDPZZXORANCNFSM4HHI3CDQ .

Hi， Can you share your code？I can‘t run this author's code. The code show error that "AttributeError: 'Seq2Seq' object has no attribute 'netName'"

[DuoCaiXie]

Hi Raymond, I’ve got the code working! I’ll send it to you. I’m surprised to learn that the training is taking so long about 31h per epoch. Is this correct? Regards, … On Mon, 10 Jun 2019 at 12:29 Raymond Zhang @.**> wrote: @.** <https://github.com/aleonnet>， I am currently using the extracted data, one file per house. It didn't uploaded to github before because these files are too large (341MB). Just now I am surprised to find that there is only 7MB after compression, so I uploaded the compressed package and committed just now, please unzip it and use it directly. I'll take the time to rewrite the generating code. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2?email_source=notifications&email_token=AE5JYV355GHSMYQGPXYACHDPZZXORA5CNFSM4HHI3CD2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODXKGUOA#issuecomment-500460088>, or mute the thread https://github.com/notifications/unsubscribe-auth/AE5JYV5VBXAL2WLQT7HDTVDPZZXORANCNFSM4HHI3CDQ .

hi，aleonnet, I want to know how you create the.pickle file for each house. Can you provide me with the code you wrote? Thank you. Regards, Wang Tao

Hi, aleonnet，

I have found the code for get_house_data.py in your repository. I am new to Github and thank you very much for providing the code.

Regards, Wang Tao

你好，请问你找到了么，能提供个链接吗