gen_epw for input files

[biglimp]

Would it be possible to add functionality to convert standard input SUEWS/UMEP met forcing files to epw-files? To me it looks like a small adjustment of gen_epw could make this happen.

[rarygit]

I use the shinyweathrdata app (https://shinyweatherdata.com/) for VCWG to generate and download ERA5 meteorological data in both CSV and EPW file formats.

The online app also provides useful Energy / Heating / Cooling index or degree days charts.

[sunt05]

Would it be possible to add functionality to convert standard input SUEWS/UMEP met forcing files to epw-files? To me it looks like a small adjustment of gen_epw could make this happen.

yes, it's possible; but the issue is the appropriateness of using standard met input for EPW: EPW needs near-surface met variables (e.g. T2, RH2, U10, etc.), which are not necessarily the same as those in the standard met files, in particular for urban met forcing data.

hence the current approach is to use the SUEWS output - which includes those near-surface diagnostics - for generating EPW.

[biglimp]

As UMEP input forcing can be non standard (of different height e.g. for SOLWEIG-modelling) this functionality is still needed. I want this functionality in order to get forcing data to e.g. the Urban Weather Generator. For example, using ERA5 data as input to UWG OR just forcing data from any standard meteorological station.

[sunt05]

I see - then perhaps you can go ahead with a look at the gen_epw source code and I believe there is something useful for making this function more adaptive and generic. If you need more help, please let me know.

[biglimp]

@rarygit, we need something in Python to go with UMEP. A web-based app is not useful, unfortunately.

[sunt05]

I use the shinyweathrdata app (https://shinyweatherdata.com/) for VCWG to generate and download ERA5 meteorological data in both CSV and EPW file formats.

The online app also provides useful Energy / Heating / Cooling index or degree days charts.

thanks @rarygit , that looks interesting and we may potentially something in supy/umep.

[biglimp]

@rarygit , would you be interested in writing something that converts metdata into epw for us?

[rarygit]

Realistically, we would be better to approach Lukas Lundström directly (aka. lukas-rokka. Mälardalen Uni, Sweden) and who developed the ERA5 / EPW app. https://confluence.ecmwf.int/pages/viewpage.action?pageId=181128871 https://github.com/lukas-rokka?tab=repositories https://www.researchgate.net/publication/325683046_Heat_demand_profiles_of_buildings%27_energy_conservation_measures_and_their_impact_on_renewable_and_resource_efficient_district_heating_systems https://scholar.google.com/citations?user=e2dYaLsAAAAJ

@biglimp - the answer is yes, but I am an apprentice programmer and I would need someone alongside as a mentor. I would start by adapting something. Maybe Ting or Nils or .... could look over my shoulder?

I use conda/spyder/ on Ubuntu to access pyqgis.core, and followed the steps described here: https://prasaz.medium.com/how-to-setup-pyqgis-with-spyder-b1459c955b97

[biglimp]

I think it would be very valuable if you started your developer career in our community. @sunt05 , @nilswallenberg and @gusbacos and me will assist you when you have questions.

What environment you use for your development is not super important as this stage. If you later move on to actually develop plugins for UMEP that you need an environment that can talk to QGIS and OSGeo Python environment

Why dont you try to make a function that converts UMEP meteorological input files to epw-files? That is a good first task for you. I think you can start by looking at gen_epw in the SuPy code. In my head it seems like just some small adjustments to that function would create such function.

[rarygit]

Ok, I will try to make a function, starting with this supy code: supy.util.gen_epw "Generate an epw file of uTMY (urbanised Typical Meteorological Year) using SUEWS simulation results" https://supy.readthedocs.io/en/latest/api/supy.util/supy.util.gen_epw.html#supy.util.gen_epw

[rarygit]

Update:

supy.util.gen_epw(df_output, lat, lon, tz=0, path_epw=Path('uTMY.epw'))

I am using the SuPy Quick Start Tutorial to generate SUEWS output for the epw file https://supy.readthedocs.io/en/latest/tutorial/quick-start.html

I added this script to the end of the run:

sp.util.gen_epw(df_output, 51.5100, -0.1200, tz=0, path_epw=Path('/home/l1nux/Documents/SuPy/sample_run/uTMY.epw'))

It threw an AttributeError: 'MultiIndex' object has no attribute 'year'

263 # calculate weighted score 264 df_output_x = df_output.assign( --> 265 Year=lambda df: df.index.year, 266 Month=lambda df: df.index.month, 267 Day=lambda df: df.index.day, 268 Hour=lambda df: df.index.hour, 269 Minute=lambda df: df.index.minute,

I guess it is expecting a 'year' column in the df_output. Looking at df_output.head(), I see the SUEWS output is "datetime", 2012-01-01 00:05:00 I am assuming the dataframe needs amending towards achieving the epw-style "calculating weighted score".

Somewhere inside here: https://github.com/UMEP-dev/SuPy/blob/master/src/supy/_save.py # save output files def save_df_output( df_output: pd.DataFrame, .... list_year = df_save.index.get_level_values("datetime").year[:-1].unique()

It seems related to EPW Issue UMEP-dev/SUEWS#152

Am I heading in the right direction?

Attached are the asciidoc and ipynb files. SuPy3_gen_epw.ipynb.zip SuPy3_gen_epw_asciidoc.zip

2022-02-10

[rarygit]

Notes as I educate myself about converting TMY to EPW in SUPy.

My starting point is to step through the SuPy script "_tmy.py" in SuPy and the module "supy.util._tmy", while being aware of the steps to create an epw file as listed below.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

There are functions in the diyepw tool developed by Pacific Northwest National Laboratory that allows the quick and easy generation of a set of epw files from WMO weather data. https://github.com/IMMM-SFA/diyepw

In particular understanding the script meteorology.py at line 152 (# Write new EPW file") https://github.com/IMMM-SFA/diyepw/blob/main/diyepw/meteorology.py This represents a time series of meteorological measurements at a given location (aka. suews output).

Similarly in the script create_amy_epw_file.py, from line 139. https://github.com/IMMM-SFA/diyepw/blob/main/diyepw/create_amy_epw_file.py This package is a tool for the generation of AMY (actual meteorological year) EPW files, which is done by injecting AMY data into TMY (typical meteorological year) EPW files.

This script has a link to laf.py (by the diyepw developers) to generate epw files from TMY and localised actual meteorlogical files (LAF). Convert to epw at line 317 (# Write new EPW file) https://github.com/SSESLab/laf/blob/master/LAF.py ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Publication in August 2021, JOSS, Journal Of Open Source. https://www.theoj.org/joss-papers/joss.03313/10.21105.joss.03313.pdf

"Some modelers have created their own weather files by obtaining weather data and manipulating it to meet the EPW format, but it is a labor-intensive process and no open-source, automated software package existed to produce EPW files from publicly available weather observations until diyepw. This software will benefit the BEM community by allowing for easy use of reliable, quality-checked, publicly available weather data in their EnergyPlus simulations to represent actual historical years in specific location(s)."

[gusbacos]

Hi,

I think you rather than using output from SUEWS could start with some UMEP-formatted input forcing data. I suggest that you use the meteorological sample data provided in supy as input, as it is already UMEP-fromatted.

Then you can read it as a pandas dataframe like this:

path = 'C:/supy/baserun/Input_init' #Change to your path 

# Read UMEP formatted txt file as pandas dataframe
UMEP_metfile = pd.read_csv(path+ '/Kc_2011_data_60.txt', delim_whitespace=True)

# fix a datetimeindex using columns in UMEP_metfile
UMEP_metfile['time'] = [dt.datetime(int(UMEP_metfile['iy'][i]),1,1)+dt.timedelta(days=int(UMEP_metfile['id'][i])-1,hours=int(UMEP_metfile['it'][i]),minutes=int(UMEP_metfile['imin'][i]))for i in range(UMEP_metfile.shape[0])]
UMEP_metfile.set_index('time', inplace = True)

UMEP_metfile

which you then can use as input in the

gen_epw(UMEP_metfile, 51.5100, -0.1200)

I would suppose that you will need to do some adjustments in the UMEP_metfile-dataframe to make it compatible with the gen_epw() function, as the columns in UMEP_metfile is not consistent with what is used in the function.

So perhaps you could create a dictionary that you can use to rename the columns in the UMEP_metfile dataframe to match what is asked for in the gen_epw()-function?

for instance in the UMEP_metfile you have 'kdown' where the gen_epw() is looking for 'Kdown'

Perhaps that is a way to start. Just let us know if you have further questions!

Best, Oskar

[rarygit]

@ Oskar, thanks for replying and giving guidance!

I was using the jupyter notebook approach for SUPy; but you are correct, I can start with the UMEP met file.

So perhaps you could create a dictionary that you can use to rename the columns in the UMEP_metfile dataframe to match what is asked for in the gen_epw()-function?

Exactly, to rename the columns; and it helps to know that creating a dictionary is a start. I now searched "dict" in _tmy.py to see where and how it is being used.

# dict: SuPy variables -> EPW standard names dict_supy_epw = { "Kdown": "Global Horizontal Radiation",

I will post further questions - but this task is being done while I am drafting a report/paper. So progress might be erratic.

[rarygit]

If I use the UMEP-formatted input forcing data, then I am using an AMY (actual met year) as input for the epw file. Is that correct?

Whereas SuPy is also interested in using TMY (typical met year) as input for the epw file.

As an aside, I have been downloading both TMY and equivalent EPW files using the TMY Tool from the EU website: PVGIS https://ec.europa.eu/jrc/en/pvgis https://re.jrc.ec.europa.eu/pvg_tools/en/#TMY

[biglimp]

Yes, correct. You can consider it an actual met-file.

[rarygit]

Thanks for that! 

[rarygit]

As attached, work-in-progress this afternoon following help from Oskar, @gusbacos Basically a combination of importing a UMEP met file ('df_umep'), then continuing the supy epw-related functions in _tmy.py. I have substituted df_umep for the occurences of TMY, plus a few other name changes.

The pvlib website was also helpful (creating a dictionary). The pvlib module is imported into the supy file "_tmy.py" and the pvlib scripts are well-annotated for reading the steps. https://github.com/pvlib/pvlib-python/blob/master/pvlib/iotools/pvgis.py

I had an error at line 139 in the attached file, "tz undefined". # assign timezone info df_umep.index = df_umep.index.tz_localize(tz * 3600) tz_localize relates to pandas.Series.tz_localize https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.tz_localize.html And assumed that tz in (tz * 3600) is the issue. Do I need to re-assign the timezone in the UMEP met file? Probably not (unless someone else differs); I think this step relates to processing the TMY file So will move on down the script.

wip1_convert_epw.zip

[biglimp]

The UMEP metdata should be in local time so the time zone should have to be included. I dont remember how timesone is represented in EPW-files but if they are, that need to be specified as an input to your translation function. Is EWP in UTC=0 or local time?

On further question is how you populate ground temperature in the header of the EPW-file?

[rarygit]

@biglimp Thanks for the timezone answer. Tentatively, I would say that the epw files are in local time, following TMY3 format. https://build.openmodelica.org/Documentation/Buildings.BoundaryConditions.WeatherData.ReaderTMY3.html

Populating the (monthly) Ground Temperatures header follows a csv file format. And would be an adaptation of the pvlib or some other equivalent pandas function.

Sources of information:

(JoeHuang): I was being cheeky because I was actually in the EnergyPlus Development Team meetings when the EPW format was developed and circulated for comment In fact, i think the EPW format is duplicative, because it just took the data in NREL's TY2 / TMY3.CSV formats and rearranged the columns. Over half of these are not needed or used in EnergyPlus, including all the illuminance values, visibility, surface albedo (?), etc. .# index 3 thru 5 are for soil conductivity, density and specific heat https://unmethours.com/question/26990/how-to-read-ground-temperature-from-epw-file/

In every epw file there are monthly ground temperatures at 3 different depths https://mostapharoudsari.gitbooks.io/ladybug-primer/content/text/components/Import_Ground_Temp.html

Ground Temperatures (Number of Ground Temperature Depths (up to 3), Depth for each Ground Temperature set, Soil Conductivity, Soil Density, Soil Specific Heat, Monthly Average Ground Temperatures) http://www.exemplary.com.au/download/data/EPW%20Weather%20Data%20Format%20Description.pdf (see page 4)

Ground Temperatures Header/Data (CSV) The results from the ground temperature heuristic calculation are shown, typically for 3 depths. Users may also fill in the blank fields (soil conductivity, soil density, soil specific heat) with known values and/or perform their own calculations and depths and supply those. These should be considered “undisturbed” ground temperatures - temperatures of soil that have not been disturbed by construction. They are not considered appropriate for calculations of building losses. The program uses a heuristic, time lagged calculation based on dry bulb temperature and location. References on the topic are found in Kusuda (see references). https://bigladdersoftware.com/epx/docs/8-2/auxiliary-programs/epw-csv-format-inout.html

Example#1 of Ground Temperatures (in csv format):

Ground parameters: # index 3 thru 5 are for soil conductivity, density and specific heat Depth for each Ground Temperature set, Soil Conductivity, Soil Density, Soil Specific Heat: 3,0.035,,,, Monthly average gound temperatures: 1st Ground Depth: 2.92,3.68,5.74,8.47,12.05,17.67,17.52,18.03,13.1,9.13,6.12,3.78,0.175,,,, 2nd Ground Depth: 2.43,3.51,5.5,7.99,11.48,16.77,17.03,17.85,13.58,9.21,6.74,4.75,0.64,,,, 3rd Ground Depth: 1.08,3.1,4.88,6.63,9.79,14.01,15.58,16.77,14.73,10.27,8.51,7.23

GROUND TEMPERATURES (csv format) 3,0.035,,,,2.92,3.68,5.74,8.47,12.05,17.67,17.52,18.03,13.1,9.13,6.12,3.78,0.175,,,,2.43,3.51,5.5,7.99,11.48,16.77,17.03,17.85,13.58,9.21,6.74,4.75,0.64,,,,1.08,3.1,4.88,6.63,9.79,14.01,15.58,16.77,14.73,10.27,8.51,7.23

Example#2 of Ground Tempratures https://bigladdersoftware.com/epx/docs/8-2/auxiliary-programs/epw-csv-format-inout.html

Number of Ground Temperature Depths,Ground Temperature Depth {m},Soil Conductivity {W/m-K},Soil Density {kg/m3},Soil Specific Heat {J/kg-K},Jan {C},Feb{C},Mar {C},Apr {C},May {C},Jun {C},Jul {C},Aug {C},Sep {C},Oct {C},Nov {C},Dec {C},

3,.5,,,,20.69,22.30,22.69,22.26,19.95,17.43,15.09,13.43,12.99,13.86,15.84,18.29,2,,,,19.18,20.71, 21.41,21.40,20.16,18.43,16.58,15.03,14.25,14.45,15.59,17.28,4,,,,18.18, 19.38,20.10,20.30,19.82,18.80,17.56,16.35,15.56,15.39,15.89,16.89

https://climate.onebuilding.org/papers/EnergyPlus_Weather_File_Format.pdf

And here is a critique of the Ground Temperature reliability by the developer (Joe Huang) of the EnergyPlus Weather Converter application: https://unmethours.com/question/29233/reliability-of-ground-temperature-data-in-epw-files/

It's always been known that these ground temperatures are very approximate. If there's a high water table or a geothermal hot spot, the temperatures could be off a lot. Also, these temperatures are meant for undisturbed soil away from any building. In truth, any construction above grade will change the ground temperature below.

[biglimp]

Well, it looks like you know how to populate ground temperature. Good! When you are happy with your function I would like to add it into the MetdataProcessor, simply by adding a choice to save as epw. Then the too, checks that the data consist of one full year and then call your function and save a epw-file. This will really be a good contribution to UMEP as we now have included the UWG but we have no good way to create input metdata for UWG.

[rarygit]

Ok, thanks for the comments.

One question I do have. From my understanding, UWG is used to morph a rural meteorological time series into an urban time series. But you already have that urban time series, as generated by SUEWS? It would seem that the conversion route would be SUEWS output --> .epw input --> EnergyPlus ?

VCWG v2 handles this situation accordingly. If you only have rural meteorological input (.epw file format), then you run the appropriate VCWG v1 algorithm (incorporating monin-obukhov/MOST) and not the UWG morphing algorithm. If you already have an urban meteorological time series, either SUEWS output or ERA5 data (ERA5 data format), then you run VCWG v2 algorithm to obtain the urban air temperature, sensible and latent heat fluxes, as well as waste heat from the building into the urban canyon.

By converting a SUEWS met data into an epw for the UWG algorithm, are we morphing an urban into an über-urban meterological scenario? In other words the morphed SUEWS urban air temperature cannot be relied on?

Rural versus Urban forcing weather data and boundary conditions, as described here: http://aaa-scientists.com/Moradi%20et%20al%201%202022.pdf

[biglimp]

We would like more options than SUEWS for this, hence UWG (and later VCWG) is good to include in UMEP. Is your function ready for testing?

[rarygit]

(1) OK, then we have one immediate option for loading an .epw into UMEP/UWG:

PVGIS 5.2 from 1 March 2022 _GöteborgLandvetterAirport As an example, I follow the example from MIT UWG, using an .epw file from an airport that can be morphed to an urban centre The dataset is ERA5 2005 - 2020, rolled into a TMY and formatted as .epw and .csv PVGIS_ERA5_Göteborg.zip It also contains Göteborg (urban)

There is an API for PVGIS, which could be incorporated into UMEP in a similar fashion to pre-processing ERA5 data using the supy/csapi functionality. https://joint-research-centre.ec.europa.eu/pvgis-photovoltaic-geographical-information-system/getting-started-pvgis/api-non-interactive-service_en

In fact, that function seems to be already made for us at the mechatronics blog and their GitHub repo, with notebook: https://mechatronicsblog.com/weather-data-solar-temperature-wind-in-python/ https://github.com/MechatronicsBlog/Weather_data_Python_PVGIS

(2) My "suews --> epw" function isn't ready, but it is getting my attention in between working out an optimal approach with Tree Planter. That means I have spyder open with the "convert to epw" script, while I solve optimisation scenarios with Tree Planter. When I have a spare moment I work on the script, which is educative, but would be quicker if I knew what I was doing.

[rarygit]

convert3_epw.zip

(i) Help needed at line 180 to define lat, lon (are they to be added to dataframe 'df_umep' ?)

solar_zenith_deg = pvlib.solarposition.get_solarposition( df_umep.index, lat, lon ).zenith Traceback (most recent call last):

Input In [79] in df_umep.index, lat, lon

NameError: name 'lat' is not defined

(ii) Also, at line 139, I have commented this out as I thought it related to TMY conversion, and not related to an AMY: Is that correct?

Otherwise I get an intractable error (at least for me):

df_umep.index = df_umep.index.tz_localize(tz * 3600) Traceback (most recent call last):

Input In [81] in df_umep.index = df_umep.index.tz_localize(tz * 3600)

NameError: name 'tz' is not defined

(iii) Similarly, at line 101. Should I expect to see file 'uAMY.epw' created with this function? or later?

++++++++++++++++++++++++++++++++++++++++++++++++++++++

.# Read UMEP formatted txt file as pandas dataframe .# Do check that your SUEWS output file has ALL the headers listed below from line 90; .# RH2, Q2 ...... .# Otherwise you will come to grief from line 143, =8-(

df_umep = pd.read_csv(path+ '/test1_2005_SUEWS_60.txt', delim_whitespace=True)

2022-03-04

[rarygit]

Have a look at the attached script. It will generate an epw file (uTMY, urban Actual Meteorological Year aka SUEWS df_output). It more closely follows the supy utility script "_tmy.py" without the TMY calculations.

There are still several errors that need to be fixed, but at least you know the script is heading in an 'epw direction'. It contains the SUEWS output meteorological file, 'test1_2005_SUEWS_60.txt', to generate the dataframe:

amy_uTMY.zip

In the next iteration I will change all references from 'tmy, TMY' to 'amy, AMY' and 'typical to 'actual' the datetime is scrambled; the values need to be corrected; the meta needs to be changed ... etc Most important is the creation of the acual epw file; next is the content.

@biglimp: the attached script needs further input and refinement from others, which I look forward to happening. Regardless whether that happens, I will slowly work through it as a learning exercise for connecting urban SUEWS output (epw) to EnergyPlus via opyplus.

2022-03-10

[rarygit]

Next iteration renamed: "wip1(work-in-progress) create actual met year" wip1_create_amy.py Source: loosely based on '_tmy.py' (SuPy) Changed: all references to "tmy, TMY, typical" changed to "amy, AMY, actual". Changed: header format, now based on https://designbuilder.co.uk/cahelp/Content/EnergyPlusWeatherFileFormat.htm Whoever uses the script has to fill in the header, but it has been made easier to do so within your Python IDE. To facilitate this I have added an EPW_Header.csv, which you can open in LibreOffice and paste in your values. Then open that csv file in a text editor and copy/paste its header text into the attached python script, from line 280. That is what I have been doing so far. For further description of the header, I have also included an .ods file with useful information.

To be done: in the epw file, the datetime format and the actual SUEWS values are incorrect. At the very least, dataframes df_amy and df_epw need to be checked and corrected.

Usually I run the script either line by line or function by function to make sure all dataframes etc are created. Ponderous, but cautious.

I have other work to continue with, but will (slowly) work on this; or follow the (quicker) improvements made by others. If it needs to be said, yes test it.

wip1_create_amy.zip

2022-03-10

[biglimp]

Just had a very quick look. Looks nice but I think there is one missunderstanding. I would like to be able to put a standard UMEP-formatted input forcing file into epw-format. I only see SUEWS output as input now, or am I missing something. I am referring to a file like the one attached. metdata_NYC_2010_data_60.txt

[rarygit]

@biglimp: thanks for replying and yes, you are right, there was a misunderstanding.

Would it be possible to add functionality to convert standard input SUEWS/UMEP met forcing files to epw-files?

Yes, it's possible; but the issue is the appropriateness of using standard met input for EPW: EPW needs near-surface met variables (e.g. T2, RH2, U10, etc.), which are not necessarily the same as those in the standard met files, in particular for urban met forcing data. Hence the current approach is to use the SUEWS output - which includes those near-surface diagnostics - for generating EPW .... perhaps you can go ahead with a look at the gen_epw source code and I believe there is something useful for making this function more adaptive and generic.

Learning by making mistakes, I started with the script '_tmy.py' and naturally followed the route of df_output --> df_epw. To see if I could get somewhere with it. At this stage it seems that we can do both approaches.

I don't think much has to change in terms of df_met --> df_epw. I have already started bringing in a UMEP met file, because I needed variables such as atmos pressure. So maybe I will continue with this approach and for now call the script "umep2epw.py"

[CsillaVG]

Hi @biglimp & @rarygit,

I just came across this old and long thread. I wonder how things are going. In case there is still a need for an "UMEP-forcing to epw-format" code let me know. I made a few 'duct-tape' functions that are not pretty but work.

Best, Csilla

[biglimp]

Sounds great. If they work we would be very happy to include them into UMEP!

[rarygit]

Hi Csilla, Short answer - yes. 20.07.2022, 17:46, "Csilla V Gal" @.>: Hi @biglimp & @rarygit,I just came across this old and long thread. I wonder how things are going. In case there is still a need for an "UMEP-forcing to epw-format" code let me know. I made a few 'duc-tape' functions that are not pretty but work.Best,Csilla—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.>

[CsillaVG]

This is a rather simple approach that builds on supy's utility, supy.util.gen_epw(). At the moment, it has some caveats (see below) that hopefully can be worked out and the approach integrated.

Attached is a zip file with an example UMEP forcing file (Budapest-Lorinc.txt), the .csv file with for converting UMEP standard climate parameter names to that of SUPY (cNames_U2S.csv) and the Python file with two functions (could be unified?) and a simple example to be run.

ASSUMPTION: The main assumption is that the provided UMEP forcing (more or less) meets the specifications of an epw file: most importantly that the Ta/RH and wind speed data refer to the standard 2 and 10 m screening heights.

CAVEATS:

1. The parameters lost in the process are precipitation, sky cover and wind direction.
2. The parameters modified are : (a) dew point temperature and atmospheric pressure, and (b) direct and diffuse shortwave radiation.

The most likely explanation for the above issues is that SuPy does not use these parameter in its calculation. Consequently, in saving SUEWS results as epw these parameters are recalculated: (a) from atmospheric variables*, and (b) by re-partitioning Kdown.

• Some differences between the input and output parameters here are also introduced with the calculation of specific humidity (which is part of my function). I used metpy for the calculation.

The other approach to the task is a bit of a tour de force, i.e. controlling all aspects of the epw file (see e.g. all the different NaN definitions used in their standard).

SimpleUMEPepwConversion.zip

[biglimp]

Cilla, can we see the code?

[CsillaVG]

Yes, it is among the zipped files (wasn't sure how I should post it here). I will copy-past the core part in a sec.

[CsillaVG]

def get_UMEPfrcg(filepath):
    '''
    Returns UMEP derived forcing data without columns containing NaN
    '''
    header = ['iy','id','it','imin','Q*','QH','QE','Qs','Qf','Wind','RH','Td','press','rain','Kdn','snow','Ldown','fcld','wuh','xsmd','lai_hr','Kdiff','Kdir','Wd']

    dataf = pd.read_table(filepath,engine='python',delimiter=' +')
    dataf.columns = header
    dataf.index = pd.to_datetime(dataf.iy.astype(str) + ' ' + dataf.id.astype(str) + ' ' + dataf.it.astype(str) + ' ' + dataf.imin.astype(str), format = "%Y %j %H %M")

    # Convert -999 to NaN & drop NaNs
    dataf.replace(-999, np.nan, inplace=True)
    dataf.dropna(axis=1,inplace=True)
    return dataf

def convert_UMEPf2epw(df_forcing,lat,lng,timeZLocal,path_save):
    '''
    Converts UNEP forcing to .epw
    '''
    df_data = df_forcing.copy()

    # Match units with SUEWS
    df_data['press'] = df_data['press']*10

    # Translate column names using table in cNames_U2S.csv
    cNames = pd.read_csv("cNames_U2S.csv",header = 0,index_col = 0,parse_dates=True)
    d = cNames.set_index('varUMEP').to_dict() 
    df_data.columns = df_data.columns.to_series().map(d['varSUEWS'])

    # Calculate specific humidity (Q2)
    temperature = df_data['T2'].values * units.degC
    relhum  = df_data['RH2'].values * units.percent
    press = df_data['pres'].values * units.hPa 

    mixing_ratio = mixing_ratio_from_relative_humidity(press,temperature,relhum)
    spec_hum = specific_humidity_from_mixing_ratio(mixing_ratio) * units('kg/kg')
    df_data["Q2"] = spec_hum.to('g/kg') 

    sp.util.gen_epw(df_data,lat,lng,timeZLocal,path_save)

[CsillaVG]

What I also meant to write, is that with some basic coding skills, one could easily fix the issues with supy.util.gen_epw and turn it into a function of its own.

[sunt05]

Thanks @CsillaVG for the nice posts!

I'd encourage you turn them into functions in supy by submitting a PR - I'll be happy to merge those into the main repo after review so your contribution can be credited.