Window sizes and quality of results

[tuhinjubcse]

In the Neural NILM paper it says https://arxiv.org/pdf/1507.06594.pdf

The window width is decided on an appliance-by-appliance basis and varies from 128 samples (13 minutes) for the kettle to 1536 samples (2.5 hours) for the dish washer

Where in your code can I find the window width ? Or if you could point me to the code where we can play with window width

[OdysseasKr]

You can set the window width when initializing the dissagregator. For example, for 128 window

d = DAEDisaggregator(128)

[tuhinjubcse]

What about the RNN ?

[tuhinjubcse]

Also while trying to train across buildings I did

train_elec1 = train.buildings[2].elec train_mains1 = train_elec1.mains() # The aggregated meter that provides the input train_meter1 = train_elec1.submeters()['fridge'] # The microwave meter that is used as a training target

train_elec2 = train.buildings[3].elec train_mains2 = train_elec2.mains() # The aggregated meter that provides the input train_meter2 = train_elec2.submeters()['fridge'] # The microwave meter that is used as a training target

train_elec3 = train.buildings[5].elec train_mains3 = train_elec3.mains() # The aggregated meter that provides the input train_meter3 = train_elec3.submeters()['fridge'] # The microwave meter that is used as a training target

train_elec4 = train.buildings[6].elec train_mains4 = train_elec4.mains() # The aggregated meter that provides the input train_meter4 = train_elec4.submeters()['fridge'] # The microwave meter that is used as a training target

train_mains = [train_mains1,train_mains2,train_mains3,train_mains4] train_meter = [train_meter1,train_meter2,train_meter3,train_meter4]

dae.train_across_buildings(train_mains, train_meter, epochs=6, sample_period=10) dae.export_model("cross_model128.h5")

The error I got is

/Desktop/neural-disaggregator/DAE/daedisaggregator.py in train_across_buildings_chunk(self, mainchunks, meterchunks, epochs, batch_size) 167 print(e) 168 batch_indexes = range(min(num_of_batches)) --> 169 random.shuffle(batch_indexes) 170 171 for bi, b in enumerate(batch_indexes):

/anaconda3/envs/nilmtk-env/lib/python3.6/random.py in shuffle(self, x, random) 275 # pick an element in x[:i+1] with which to exchange x[i] 276 j = randbelow(i+1) --> 277 x[i], x[j] = x[j], x[i] 278 else: 279 _int = int

TypeError: 'range' object does not support item assignment

[tuhinjubcse]

I commented shuffle and this got resolved , however I do not understand why if train across buildings , my MAE increases while it should decrease as the neural net is getting a lot more data

[tuhinjubcse]

Can you reply to this ? These are my results

[OdysseasKr]

Sorry for the late reply.

1. What about the RNN ?

As mentioned in the paper by J. Kelly et al

At each time step, the network sees a single sample of aggregate power data and outputs a single sample of power data for the target appliance. thus the RNN only has a window width of 1.

2. TypeError: 'range' object does not support item assignment

I am guessing that you are using Python 3. You need to change this

batch_indexes = range(min(num_of_batches))

to this

batch_indexes = list(range(min(num_of_batches)))

Usually shuffling helps with training so it's a good idea to keep it. At some point I'll try to make the code compatible with Python 3 and this error will be fixed.

3. if train across buildings , my MAE increases while it should decrease as the neural net is getting a lot more data

Since lots of factors go into getting better results (not just the amount of data) I can not really comment on that.

4. Can you reply to this ? The range on the time-axis is too wide to actually see the activations. Try to plot an hour of data or so in order to be able to compare.

Please try to use discriptive issue titles. Also create new issues if the new question is unrelated. We need to keep the issues clear enough for other people to reference, as well.

[tuhinjubcse]

Hey i think got a bug in DAE train_accross_building_chunks maybe thatswhy bad results

for i in range(num_meters): mainpart = mainchunks[i] meterpart = mainchunks[i] //this should be meterchunks mainpart = mainpart[bbatch_sizes:(b+1)batch_sizes] meterpart = meterpart[bbatch_sizes:(b+1)batch_sizes] X = np.reshape(mainpart, (batch_size, s, 1)) Y = np.reshape(meterpart, (batch_size, s, 1))

                X_batch[i*batch_size:(i+1)*batch_size] = np.array(X)
                Y_batch[i*batch_size:(i+1)*batch_size] = np.array(Y)

[OdysseasKr]

Hey i think got a bug in DAE train_accross_building_chunks maybe thatswhy bad results

Thank you for that. Just fixed it, should be correct now. It shouldn't have affected your results unless you trained on multiple buildings using the train_across_buildings method.