old ascii ED2 met driver file as example?

[elizsgarcia]

I have met files in ascii format that I'd like to convert to hdf5 using the convert_ascii.f90 program provided with the ED2 download. Because the file is set up to read an old-style ED2 met driver file, I'd like to format my ascii file to match that format. Does anyone have an old ascii ED met driver file that they can share? Or a screenshot of what it looks like? I found a tutorial that named "filled_santarem_km83.txt" as an example.

Thanks for any help!

[mdietze]

Let's step back -- if the met you have isn't in the ED2 ascii met format, what is the source of the data and what format is it in? I'm guessing your question is really about how you build an ED2 met file, and there are a bunch of ways that will be easier than the route you're proposing (and there's a nontrivial chance that one of us might have code for it already)

[elizsgarcia]

The data is in an Ameriflux format -- please see the screenshot of the first several lines of the file below. I'm very open to other options for building the met files. Thanks for your help!

[mdietze]

PEcAn (pecanproject.org, github.com/PecanProject/pecan) can process Ameriflux L2 automatically, including download, gap-filling, and conversion to model-specific formats (including but not limited to ED2), all in a 'behind the scenes' workflow if you're using PEcAn from it's web interface.

If you prefer to do it manually, then you can use the PEcAn.data.atmosphere and PEcAn.ED2 R packages (located in modules and models respectively). The basic workflow is that PEcAn converts all met to netCDF with variable names following the CF convention -- in this case you'd use met2CF.Ameriflux if you're starting with Ameriflux L2 or met2CF.csv for any other generic csv data. Next you'd use metgapfill for gap filling and then met2model.ED2 to convert to ED's format (both the hdf5 files and the header file).

FYI, PEcAn has converters for a few other common met formats (e.g. NARR) and more are under development, as are standardized tools for temporal and spatial downscaling and dealing with uncertainties. The original PEcAn met workflow was developed my my student, @bcow, but this all will be lead by a new student in @ankurdesai 's lab.

[robkooper]

Finally you can use Brown Dog (with help from PEcAn) to do all the conversions for you. All you have to do is upload a small xml file and you can download the processed file for ED. See http://browndog.ncsa.illinois.edu/ for more information. Feel free to ask @mdietze, @mchenry or @kooper for more information about Brown Dog.

[elizsgarcia]

Thank you for your suggestions, I'll try using PEcAN.

[fabeit]

@mdietze @robkooper I need to convert either GRIB or NETcdf to hdf5 for ED. I looked at polyglot but I can't figure out how it works, is it a service that runs on a server with a browser interface? Will met2model work directly with GRIB or netcdf or does it need .csv?

[mdietze]

met2model.[MODEL] specifically converts from netCDF files following CF metadata to model-specific drivers. These are located in the PEcAn model directory https://github.com/PecanProject/pecan/tree/master/models

To get things into netCDF files, you can use the met2CF.[format] functions in https://github.com/PecanProject/pecan/tree/master/modules/data.atmosphere/R

These functions are part of the larger PEcAn met.process workflow https://github.com/PecanProject/pecan/blob/master/modules/data.atmosphere/R/met.process.R

The functions are also installed on Brown Dog, so you can use that without needing the workflow.

@bcow could pass you the draft manuscript on all of this

[fabeit]

@mdietze I have tried the DAP bookmarklet in brown dog but when I click on it a text box pops up with the message "authorization". I can get netcdf directly from ERA, so i only need to use met2model.ED2, correct?

[mdietze]

If ERA is in CF then you only need met2model.ED2, otherwise you need to convert variable names and units.

[mchenry]

@fabeit BrownDog is just about to enter a friendly user mode. You can go here to sign up and get an account:

http://browndog.ncsa.illinois.edu/signup/

We can then contact you with details as to how to begin playing with the data manipulation/analysis services.

[ch1eroe1]

@fabeit I am looking into this issue for you. Is there a way you can get me a copy of the file for testing? If you don't want send me the file publicly then let me know how you would like to get me the file.

Gene

[fabeit]

@ch1eroe1 thanks, let me know how I can send you a file. I have received an email from Kenton but the URL provided doesn't really let me log in http://dts.ncsa.illinois.edu/ . Can you help? Thanks