Closed dengjinyu123 closed 2 years ago
Henry-2000
commented
2 years ago Hi there. By coincidence, I added a comment on 'cnn.py' giving instructions on how to use GPU with Kaggle. But I advise to clone the whole repository again, cause i made some changes in saving directories and variable names. For this, I apologize. Any difficulties let me know.
dengjinyu123
commented
2 years ago Thank you so much, you are so kind
------------------ 原始邮件 ------------------ 发件人: "Henry-2000/NILM_with_PLAID_dataset" @.>; 发送时间: 2022年4月11日(星期一) 晚上8:26 @.>; @.**@.>; 主题: Re: [Henry-2000/NILM_with_PLAID_dataset] dear professor how to train with gpu (Issue #2)
Hi there. By coincidence, I added a comment on 'cnn.py' giving instructions on how to use GPU with Kaggle. But I advise to clone the whole repository again, cause i made some changes in saving directories and variable names. For this, I apologize. Any difficulties let me know.
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
dengjinyu123
commented
2 years ago dear professor, I want to know your code's references,and want to know the specific drawing method of vi images. Best wishes
------------------ 原始邮件 ------------------ 发件人: "Henry-2000/NILM_with_PLAID_dataset" @.>; 发送时间: 2022年4月11日(星期一) 晚上8:26 @.>; @.**@.>; 主题: Re: [Henry-2000/NILM_with_PLAID_dataset] dear professor how to train with gpu (Issue #2)
Hi there. By coincidence, I added a comment on 'cnn.py' giving instructions on how to use GPU with Kaggle. But I advise to clone the whole repository again, cause i made some changes in saving directories and variable names. For this, I apologize. Any difficulties let me know.
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
Henry-2000
commented
2 years ago Hi there. Soon I will be uploading the related work (although it's in portuguese, I'm from Brazil) that clarifies the code's procedures (event detection and load classification). Just waiting to be published at my university. But I can advance some of the main references that were used: V-I trajectories: https://www.sciencedirect.com/science/article/abs/pii/S0378778817312690 https://ieeexplore.ieee.org/document/8580416 S-transform https://ieeexplore.ieee.org/document/1284896 https://ieeexplore.ieee.org/document/5184926 Kalman Filter https://www.mdpi.com/1996-1073/11/11/2918 https://ieeexplore.ieee.org/document/1270401 The methods used to plot V-I trajectories comes with some preprocessing steps (harmonics filtering, mean phase lag, resistive loads scaling...) that are indeed hard to understand only with code. I'll ask for a little more patience until the work is published and I'm allowed to upload in the repository. Would say a week or two, but not sure. Thanks for the interest.
dengjinyu123
commented
2 years ago thank you very much！
------------------ 原始邮件 ------------------ 发件人: "Henry-2000/NILM_with_PLAID_dataset" @.>; 发送时间: 2022年4月12日(星期二) 晚上6:19 @.>; @.**@.>; 主题: Re: [Henry-2000/NILM_with_PLAID_dataset] dear professor how to train with gpu (Issue #2)
Hi there. Soon I will be uploading the related work (although it's in portuguese, I'm from Brazil) that clarifies the code's procedures (event detection and load classification). Just waiting to be published at my university. But I can advance some of the main references that were used: V-I trajectories: https://www.sciencedirect.com/science/article/abs/pii/S0378778817312690 https://ieeexplore.ieee.org/document/8580416 S-transform https://ieeexplore.ieee.org/document/1284896 https://ieeexplore.ieee.org/document/5184926 Kalman Filter https://www.mdpi.com/1996-1073/11/11/2918 https://ieeexplore.ieee.org/document/1270401 The methods used to plot V-I trajectories comes with some preprocessing steps (harmonics filtering, mean phase lag, resistive loads scaling...) that are indeed hard to understand only with code. I'll ask for a little more patience until the work is published and I'm allowed to upload in the repository. Would say a week or two, but not sure. Thanks for the interest.
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
dengjinyu123
commented
2 years ago Dear seniors, I'm really sorry to bother you again. If I want to add the power characteristics of the corresponding electrical appliances to the vi Image Features, how can I achieve this? Best wisher
------------------ 原始邮件 ------------------ 发件人: "Henry-2000/NILM_with_PLAID_dataset" @.>; 发送时间: 2022年4月12日(星期二) 晚上6:19 @.>; @.**@.>; 主题: Re: [Henry-2000/NILM_with_PLAID_dataset] dear professor how to train with gpu (Issue #2)
Hi there. Soon I will be uploading the related work (although it's in portuguese, I'm from Brazil) that clarifies the code's procedures (event detection and load classification). Just waiting to be published at my university. But I can advance some of the main references that were used: V-I trajectories: https://www.sciencedirect.com/science/article/abs/pii/S0378778817312690 https://ieeexplore.ieee.org/document/8580416 S-transform https://ieeexplore.ieee.org/document/1284896 https://ieeexplore.ieee.org/document/5184926 Kalman Filter https://www.mdpi.com/1996-1073/11/11/2918 https://ieeexplore.ieee.org/document/1270401 The methods used to plot V-I trajectories comes with some preprocessing steps (harmonics filtering, mean phase lag, resistive loads scaling...) that are indeed hard to understand only with code. I'll ask for a little more patience until the work is published and I'm allowed to upload in the repository. Would say a week or two, but not sure. Thanks for the interest.
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
Henry-2000
commented
2 years ago Hi there. Not bothering at all. For power feature in V-I trajectories, you can use the vertical distance between highest and lowest point in the current axis. This represents active power, as it is proportional to current magnitude of loads. Take a look at https://ieeexplore.ieee.org/document/6575197. They call this feature 'span'.
dengjinyu123
commented
2 years ago Dear seniors, I'm really sorry to bother you again. Due to my weak foundation, I don't understand the data preprocessing part very well. Do you use Fryze theory to process current? Best wishes
------------------ 原始邮件 ------------------ 发件人: "Henry-2000/NILM_with_PLAID_dataset" @.>; 发送时间: 2022年4月13日(星期三) 上午8:38 @.>; @.**@.>; 主题: Re: [Henry-2000/NILM_with_PLAID_dataset] dear professor how to train with gpu (Issue #2)
Hi there. Not bothering at all. For power feature in V-I trajectories, you can use the vertical distance between highest and lowest point in the current axis. This represents active power, as it is proportional to current magnitude of loads. Take a look at https://ieeexplore.ieee.org/document/6575197. They call this feature 'span'.
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
Henry-2000
commented
2 years ago Hi there. In the code, the preprocessing of V-I trajectory images follows three steps:
That said, instead of separating current signal into its active and reactive components (as in Fryze power theory), the code uses decomposition of current signal into its harmonic components.
Note: I have put the link of related work in README.md (it's in portuguese, but the figures might help).
Best regards.
dengjinyu123
commented
2 years ago thanks for your answer，but I have two new questions，In what proportion are the training set and test set samples of the vi trajectory graph divided? The other is if I want to reduce the number of samples in the training set, where should I modify the code? best wishes
------------------ 原始邮件 ------------------ 发件人: "Henry-2000/NILM_with_PLAID_dataset" @.>; 发送时间: 2022年4月27日(星期三) 晚上8:27 @.>; @.**@.>; 主题: Re: [Henry-2000/NILM_with_PLAID_dataset] dear professor how to train with gpu (Issue #2)
In the code, the preprocessing of V-I trajectory images follows three steps:
- *Harmonics filtering*: the voltage and current data of PLAID dataset
comes with some noise (specially for current signals) that gives the effect
of blur when plotting the V-I trajectories. To soften this effect, the
function 'filter_harmonics' of harmonics.py is used to extract the
fundamental component of voltage signal and even/odd harmonics (until the
21th order) of current signal;
- *Mean lag for appliances categories*: for each category of appliances
(air conditioners, fans, microwaves,...), the current and voltage signals
in the dataset also comes with inconsistent values of phase between them
(this drastically changes the shape of trajectories). So for each category,
a mean value of phase (between voltage and current) is calculated and
applied in order to standardize the images of the same category;
- *Current scaling for resistive loads*: since resistive loads in
general creates similar V-I trajectories (same phase for current and
voltage signals, in addition to being pure sinusoidal), the current signals
for these loads are scaled taking as reference the maximum current
magnitude (that in this case belongs to the 'Water Kettle' category). This
result in images of 'Water Kettle' category being approximately a diagonal
straight line, and other resistive load (such as 'Incandescent Light Bulb'
and 'Coffee Maker' for example) being also a straight line, but shorter and
leaning more to horizontal, depending of their current magnitudes.That said, instead of separating current signal into its active and reactive components (as in Fryze power theory), the code uses decomposition of current signal into its harmonic components.
Note: I have put the link of related work in README.md (it's in portuguese, but the figures might help).
Best regards.
<https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail> Livre de vírus. www.avast.com <https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>. <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
Em ter., 26 de abr. de 2022 às 12:32, dengjinyu123 @.***> escreveu:
> Dear seniors, > I'm really sorry to bother you again. > Due to my weak foundation, I don't understand the data preprocessing part > very well. Do you use Fryze theory to process current? > > > > > > > > > > > > Best wishes > > > > > > ------------------ 原始邮件 ------------------ > 发件人: "Henry-2000/NILM_with_PLAID_dataset" @.>; > 发送时间: 2022年4月13日(星期三) 上午8:38 > @.>; > @.**@.>; > 主题: Re: [Henry-2000/NILM_with_PLAID_dataset] dear professor how to > train with gpu (Issue #2) > > > > > > > Hi there. Not bothering at all. For power feature in V-I trajectories, you > can use the vertical distance between highest and lowest point in the > current axis. This represents active power, as it is proportional to > current magnitude of loads. Take a look at > https://ieeexplore.ieee.org/document/6575197. They call this feature > 'span'. > > — > Reply to this email directly, view it on GitHub, or unsubscribe. > You are receiving this because you authored the thread.Message ID: > @.> > > — > Reply to this email directly, view it on GitHub > <https://github.com/Henry-2000/NILM_with_PLAID_dataset/issues/2#issuecomment-1109944032>, > or unsubscribe > <https://github.com/notifications/unsubscribe-auth/AR7BDNESFTG5RNCGXFTKHNLVHAEBTANCNFSM5TBWV33A> > . > You are receiving this because you modified the open/close state.Message > ID: @.> >
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
Henry-2000
commented
2 years ago Hi there.
Question 1: training set and test set were split using Stratified K-Fold method with k=4. That means a proportion of 1:4 for test and training sets (25% - test, 75% training). Question 2: in the "cnn.py" file, inside 'process_data_VI_Images' function, line 253, you can reduce the number of splits from 4 to 3 or 2 (minimum is 2). Then the proportion will be 1:3 (33.33% - test and 66.66%-training ) or 1:2 (50/50). I chose 4 because one of the appliances ('Blender' if I remember correctly) has a limited number of samples (just 8 samples). So choosing a number of splits of 3 will give an asymmetric proportion of training and test sets. But 2 splits should be ok too, I think.
Let me know if you have any more questions.
Best regards.
Em ter., 11 de out. de 2022 às 04:57, dengjinyu123 @.***> escreveu:
thanks for your answer，but I have two new questions，In what proportion are the training set and test set samples of the vi trajectory graph divided? The other is if I want to reduce the number of samples in the training set, where should I modify the code? best wishes
------------------ 原始邮件 ------------------ 发件人: "Henry-2000/NILM_with_PLAID_dataset" @.>; 发送时间: 2022年4月27日(星期三) 晚上8:27 @.>; @.**@.>; 主题: Re: [Henry-2000/NILM_with_PLAID_dataset] dear professor how to train with gpu (Issue #2)
In the code, the preprocessing of V-I trajectory images follows three steps:
- Harmonics filtering: the voltage and current data of PLAID dataset comes with some noise (specially for current signals) that gives the effect of blur when plotting the V-I trajectories. To soften this effect, the function 'filter_harmonics' of harmonics.py is used to extract the fundamental component of voltage signal and even/odd harmonics (until the 21th order) of current signal;
- Mean lag for appliances categories: for each category of appliances (air conditioners, fans, microwaves,...), the current and voltage signals in the dataset also comes with inconsistent values of phase between them (this drastically changes the shape of trajectories). So for each category, a mean value of phase (between voltage and current) is calculated and applied in order to standardize the images of the same category;
- Current scaling for resistive loads: since resistive loads in general creates similar V-I trajectories (same phase for current and voltage signals, in addition to being pure sinusoidal), the current signals for these loads are scaled taking as reference the maximum current magnitude (that in this case belongs to the 'Water Kettle' category). This result in images of 'Water Kettle' category being approximately a diagonal straight line, and other resistive load (such as 'Incandescent Light Bulb' and 'Coffee Maker' for example) being also a straight line, but shorter and leaning more to horizontal, depending of their current magnitudes.
That said, instead of separating current signal into its active and reactive components (as in Fryze power theory), the code uses decomposition of current signal into its harmonic components.
Note: I have put the link of related work in README.md (it's in portuguese, but the figures might help).
Best regards.
Livre de vírus. www.avast.com < https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail>.
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
Em ter., 26 de abr. de 2022 às 12:32, dengjinyu123 @.***> escreveu:
> Dear seniors, > I'm really sorry to bother you again. > Due to my weak foundation, I don't understand the data preprocessing part > very well. Do you use Fryze theory to process current? > > > > > > > > > > > > Best wishes > > > > > > ------------------ 原始邮件 ------------------ > 发件人: "Henry-2000/NILM_with_PLAID_dataset" @.>; > 发送时间: 2022年4月13日(星期三) 上午8:38 > @.>; > @.**@.>; > 主题: Re: [Henry-2000/NILM_with_PLAID_dataset] dear professor how to > train with gpu (Issue #2) > > > > > > > Hi there. Not bothering at all. For power feature in V-I trajectories, you > can use the vertical distance between highest and lowest point in the > current axis. This represents active power, as it is proportional to > current magnitude of loads. Take a look at > https://ieeexplore.ieee.org/document/6575197. They call this feature > 'span'. > > — > Reply to this email directly, view it on GitHub, or unsubscribe. > You are receiving this because you authored the thread.Message ID: > @.***> > > — > Reply to this email directly, view it on GitHub > < https://github.com/Henry-2000/NILM_with_PLAID_dataset/issues/2#issuecomment-1109944032>,
> or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AR7BDNESFTG5RNCGXFTKHNLVHAEBTANCNFSM5TBWV33A>
> . > You are receiving this because you modified the open/close state.Message > ID: @.***> >
— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>
— Reply to this email directly, view it on GitHub https://github.com/Henry-2000/NILM_with_PLAID_dataset/issues/2#issuecomment-1274253452, or unsubscribe https://github.com/notifications/unsubscribe-auth/AR7BDNBEZISRQOMFLZRBNX3WCUMXTANCNFSM5TBWV33A . You are receiving this because you modified the open/close state.Message ID: @.***>
I added some code using cuda myself, but still not using gpu