search

nilmtk / nilm_metadata

A schema for modelling meters, measurements, appliances, buildings etc
http://nilm-metadata.readthedocs.org
Apache License 2.0
49 stars 47 forks source link

Writing a data converter - multiple files from 1 meter #20

Closed gjwo closed 9 years ago

gjwo commented 9 years ago

I have a collection of data from a whole house meter in multiple files, with each file containing 2 columns containing, a timestamp string and an integer metric (active or reactive power). Each file contains only one type of metric and are differentiated by standardised names. Each file contains multiple days worth of 1 second granularity data, and there are many files covering a wider span of time. the files come in pairs for example 4-POWER_REAL_FINE 2013-11-20 Dump.csv <-> 5-POWER_REACTIVE_STANDARD 2013-11-20 Dump.csv 4-POWER_REAL_FINE 2013-11-28 Dump.csv <-> 5-POWER_REACTIVE_STANDARD 2013-11-28 Dump.csv 4-POWER_REAL_FINE 2013-12-01 Dump.csv <-> 5-POWER_REACTIVE_STANDARD 2013-12-01 I have already written the code to walk the directory tree and find the matching pairs. I have followed the structure that has datasetname/building1/elec Question 1 should I add another layer of ../meter1 to allow for the possibility in the future of having more meters so I can keep the files for each meter separate? Question 2 is there any scheme for incrementally adding data, that should be included in a converter or would it always read in all the data and construct a new HDF5 file?

gjwo commented 9 years ago

wrong place

JackKelly commented 9 years ago

(for anyone else following this, this discussion has moved to https://github.com/nilmtk/nilmtk/issues/395 )