search

NREL / nsrdb

This repository contains all of the methods for the NSRDB data processing pipeline.
https://nrel.github.io/nsrdb/
BSD 3-Clause "New" or "Revised" License
5 stars 0 forks source link

NSRDB Meta Data NaN values and Dtypes #27

Closed grantbuster closed 2 years ago

grantbuster commented 2 years ago

Why this feature is necessary: NaN values in string columns should be converted to str("None"), Timezone should be integer, Elevation should be float32 or int32. These fixes will make export to HSDS easier.

grantbuster commented 2 years ago

We should probably make an nsrdb utility function that does this cleaning and then use that method in two places:

https://github.com/NREL/nsrdb/blob/b2f01d3afc37b142766de4bcafb5f60b5ab2641b/nsrdb/nsrdb.py#L134 https://github.com/NREL/nsrdb/blob/b2f01d3afc37b142766de4bcafb5f60b5ab2641b/nsrdb/data_model/data_model.py#L186

bnb32 commented 2 years ago

Added clean_meta function to file_utils.py and use the function in data_model.py at https://github.com/NREL/nsrdb/blob/b5ca9cc1da8fcfc598f6d647120e49a7f972cc62/nsrdb/data_model/data_model.py#L201 and https://github.com/NREL/nsrdb/blob/b5ca9cc1da8fcfc598f6d647120e49a7f972cc62/nsrdb/data_model/data_model.py#L204 and in nsrdb.py at https://github.com/NREL/nsrdb/blob/b5ca9cc1da8fcfc598f6d647120e49a7f972cc62/nsrdb/nsrdb.py#L135