search

demand-consults / demand_acep

A data-pipeline for high-resolution power meter data
MIT License
1 stars 4 forks source link

Use selected NetCDF file reader to read files according to meter and measurement type #23

Closed aidowu closed 5 years ago

aidowu commented 5 years ago

Parse NetCDF data in preparation for loading into database

aidowu commented 5 years ago

I have written code that:

  1. Loads the "Copy of measured channel xlsx" file and reads the meter name and measurement type
  2. Goes through all the folders in the data and extracts .nc files
  3. Determines the meter name and the measurement type for each .nc file.
  4. Combines all measurements for each meter into a single data dataframe.

I implemented the above for a single day. Eventually, we will want to load the data into a database but I put in in a dataframe for now to test the functionality of the code.

aidowu commented 5 years ago

To read the .nc file, I found a package called xarray It was easier to use and has more features than the Netcdfhandler.py file.

chintanp commented 5 years ago

The NetCDF files are being read in the extract_data() function as seen here