search

nilmtk / nilm_metadata

A schema for modelling meters, measurements, appliances, buildings etc
http://nilm-metadata.readthedocs.org
Apache License 2.0
49 stars 47 forks source link

Issues with converting iawe metadata #7

Closed nipunbatra closed 10 years ago

nipunbatra commented 10 years ago

Stacktrace

/home/nipun/git/nilmtk/nilmtk/dataset_converters/iawe/convert_iawe.pyc in convert_iawe(iawe_path, hdf_filename)
     69     store.close()
     70     convert_yaml_to_hdf5(join(_get_module_directory(), 'metadata'),
---> 71                          hdf_filename)
     72 
     73     print("Done converting iAWE to HDF5!")

/home/nipun/git/nilm_metadata/nilm_metadata/convert_yaml_to_hdf5.pyc in convert_yaml_to_hdf5(yaml_dir, hdf_filename)
     51         elec_meters = building_metadata['elec_meters']
     52         _deep_copy_meters(elec_meters)
---> 53         _set_data_location(elec_meters, building)
     54         _sanity_check_meters(elec_meters, meter_devices)
     55         _sanity_check_appliances(building_metadata)

/home/nipun/git/nilm_metadata/nilm_metadata/convert_yaml_to_hdf5.pyc in _set_data_location(elec_meters, building)
     85     for meter_instance in elec_meters:
     86         data_location = '/{:s}/elec/meter{:d}'.format(building, meter_instance)
---> 87         elec_meters[meter_instance]['data_location'] = data_location
     88 
     89 def _sanity_check_meters(meters, meter_devices):

building1.yaml

instance: 1   # this is the first building in the dataset
original_name: house_1   # original name from REDD dataset
elec_meters:
  1: &EM6400
    site_meter: true
    device_model: EM6400
  2: *EM6400
  3: &jplug
    submeter_of: 0 
    device_model: jplug
  4: *jplug
  5: *jplug
  6: *jplug
  7: *jplug
  8: *jplug
  9: *jplug
  10: *jplug
  11: &current cost

appliances:
- original_name: fridge
  type: fridge
  #floor: 0
  instance: 1
  meters: [3]

- original_name: air conditioner
  type: air conditioner
  instance: 1
  #floor: 1
  meters: [4]

- original_name: air conditioner
  type: air conditioner
  instance: 2
  #floor: 1
  meters: [5]

- original_name: washing machine
  type: washing machine
  instance: 1
  #floor: 1
  meters: [6]

- original_name: laptop computer
  #floor: 1
  type: computer   
  instance: 1
  meters: [7]

- original_name: clothes iron
  #floor: 1
  type: iron
  instance: 1  
  meters: [8]

- original_name: kitchen outlets
  instance: 1  
  meters: [9]

- original_name: television
  type: television
  instance: 1
  #floor: 0
  meters: [10]

- original_name: water filter
  type: water filter
  instance: 1
  meters: [11]

- original_name: water motor
  type: motor
  instance: 1
  meters: [12]

meter_devices.yaml


EM6400:
  model: EM6400
  manufacturer: Schneider Electric
  manufacturer_url: http://www.schneider-electric.com/
  description: >
    Multifunction meter for feeders
  sample_period: 1   # the interval between samples. In seconds.
  measurements:
  - physical_quantity: power   # power, voltage, energy, current?
    type: active   # active (real power), reactive or apparent?
  - physical_quantity: power   # power, voltage, energy, current?
    type: apparent   # active (real power), reactive or apparent?
  - physical_quantity: power   # power, voltage, energy, current?
    type: reactive   # active (real power), reactive or apparent?
  - physical_quantity: frequency   # power, voltage, energy, current?
  - physical_quantity: voltage   # power, voltage, energy, current?
  - physical_quantity: energy   # power, voltage, energy, current?
    type: apparent   # active (real power), reactive or apparent?
  - physical_quantity: power factor   # power, voltage, energy, current?
  - physical_quantity: phase angle   # power, voltage, energy, current?
  wireless: false

jplug:
  description: >
  sample_period: 1
  measurements:
  - physical_quantity: power
    type: apparent
    lower_limit: 0
  wireless: true

current cost:
  description: >
  sample_period: 6
  measurements:
  - physical_quantity: power   # power, voltage, energy, current?
    type: active   # active (real power), reactive or apparent?
  - physical_quantity: power   # power, voltage, energy, current?
    type: apparent   # active (real power), reactive or apparent?
  - physical_quantity: power   # power, voltage, energy, current?
    type: reactive   # active (real power), reactive or apparent?
  - physical_quantity: frequency   # power, voltage, energy, current?
  - physical_quantity: voltage   # power, voltage, energy, current?
  - physical_quantity: energy   # power, voltage, energy, current?
    type: apparent   # active (real power), reactive or apparent?
  - physical_quantity: power factor   # power, voltage, energy, current?
  - physical_quantity: phase angle   # power, voltage, energy, current?
  - physical_quantity: current   # power, voltage, energy, current?
    type: apparent   # active (real power), reactive or apparent?
  wireless: true

dataset.yaml

name: iAWE
long_name: Indian dataset for ambient, water and electricity sensing
creators:
- Batra, Nipun
- Gulati, Manoj
- Singh, Amarjeet
- Srivastava, Mani
publication_date: 2013
institution: Indraprastha Institute of Information Technology Delhi (IIITD)
contact: nipunb@iiitd.ac.in
description: 73 days of ambient, water and electricity data for a home in Delhi
subject: First dataset from a developing country
number_of_buildings: 1
timezone: Asia/Kolkata
geo_location:
  locality: Delhi   # village, town, city or state
  country: IN  # standard two-letter country code defined by ISO 3166-1 alpha-2
  latitude: 28.64 # 
  longitude: 77.11
related_documents:
- http://iawe.github.io
- >
  Nipun Batra and Manoj Gulati and Amarjeet Singh and Mani Srivastava
  It's Different: Insights into home energy consumption in India.
  In proceedings of the 5th ACM Workshop On Embedded Systems 
  For Energy-Efficient Buildings (Buildsys 2013)
  http://nipunbatra.github.io/downloads/files/buildsys_2013.pdf
schema: https://github.com/nilmtk/nilm_metadata/tree/v0.2
JackKelly commented 10 years ago

Great work converting iAWE to NILM Metadata!!! I hope it wasn't too painful ;)

is there some of the stack trace missing??

Also, I can see some small errors in the YAML.

for example:

11: &current cost

And several like this:

  description: >

(if you have no description then just don't include the field! Or delete the ">").

nipunbatra commented 10 years ago

Is &current_cost ok? I removed the empty descriptions and changed all occurences of current cost to current_cost.

Pardon my yaml novice skills. I just tried to copy stuff from REDD metadata.

Now, I get the following stack trace

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-10-68f3babaea88> in <module>()
----> 1 convert_iawe.convert_iawe("/home/nipun/Copy/Dataset/", "/home/nipun/Desktop/dataset_123.h5")

/home/nipun/git/nilmtk/nilmtk/dataset_converters/iawe/convert_iawe.pyc in convert_iawe(iawe_path, hdf_filename)
     69     store.close()
     70     convert_yaml_to_hdf5(join(_get_module_directory(), 'metadata'),
---> 71                          hdf_filename)
     72 
     73     print("Done converting iAWE to HDF5!")

/home/nipun/git/nilm_metadata/nilm_metadata/convert_yaml_to_hdf5.pyc in convert_yaml_to_hdf5(yaml_dir, hdf_filename)
     51         elec_meters = building_metadata['elec_meters']
     52         _deep_copy_meters(elec_meters)
---> 53         _set_data_location(elec_meters, building)
     54         _sanity_check_meters(elec_meters, meter_devices)
     55         _sanity_check_appliances(building_metadata)

/home/nipun/git/nilm_metadata/nilm_metadata/convert_yaml_to_hdf5.pyc in _set_data_location(elec_meters, building)
     85     for meter_instance in elec_meters:
     86         data_location = '/{:s}/elec/meter{:d}'.format(building, meter_instance)
---> 87         elec_meters[meter_instance]['data_location'] = data_location
     88 
     89 def _sanity_check_meters(meters, meter_devices):

TypeError: 'NoneType' object does not support item assignment
JackKelly commented 10 years ago

Is &current_cost ok? I removed the empty descriptions and changed all occurences of current cost to current_cost.

I'm not sure why you want to include &current_cost is you metadata? Are you using a Current Cost meter? The & symbol is just used in YAML to label an object that you want to refer to later.

The wikipedia page on YAML is a pretty concise summary of the syntax: http://en.wikipedia.org/wiki/YAML

I think this will fix the error you're getting. I think the error is caused by the fact that meter 11 isn't a dict (which is what it should be ;)

nipunbatra commented 10 years ago

Are you using a Current Cost meter?

Ya, I am! For 1 of the 12 appliance level load which required clamping!

The & symbol is just used in YAML to label an object that you want to refer to later.

As I had thought from previous usage of pointers in C/C++.

I think this will fix the error you're getting. I think the error is caused by the fact that meter 11 isn't a dict (which is what it should be ;)

Fixed it. I think specifying a detailed metadata would be non-trivial when done the first time.

JackKelly commented 10 years ago

Should we close this issue if it's fixed?