Organize netcdf files

[dlebauer]

The netcdf files are disorganized and split up, which fails to take advantage of the format. We need to document the data and combine data sets into a single file that is easy (quick and logical) to read.

The data are here: https://github.com/ebimodeling/ghgvc/tree/master/netcdf

Presumably the metadata are in app/controllers/workflows_controller.rb.

TODO:

1. concatenating and appending data. perhaps separate by source, but it makes more sense to combine all variables into a single folder, and the separation of crops into US and Brazil is redundant with the map and lat/lon coordinates.
2. stored data separately from source code (and perhaps remove from history to reduce checkout size)
3. re-grid similar datasets, combine into single file
4. add species / pft as a dimension, see CF discussion http://kitt.llnl.gov/trac/ticket/99 e.g
   • 'taxon_identifier' as standard_name; --> scientific name

[dlebauer]

@teixeirak are values in the brazil_bare_soyb related to bare soy, or bare land generally? If related to soy, what is the relationship?

[teixeirak]

This is related to soy specifically. It is the bare ground baseline used to calculate the change in the surface energy balance if you were to clear the ecosystem. In theory, it should be the same as the global bare map, but I'm not sure it is (slightly different modeling platform- AgroIBIS as opposed to IBIS).

[dlebauer]

Okay. Got it.

[dlebauer]

List of combined files are here: https://gist.github.com/dlebauer/7cc778317e45bb36a74b

@hmb1 did the following

• combined files with multiple variables into a subset of files
  • here is a list of files and variables https://gist.github.com/dlebauer/7cc778317e45bb36a74b
  • previously, each lookup of a value required the same find nearest neighbor app/controllers/workflows_controller.rb

[hmb1]

Hi David, uploaded yesterday to my user on pecandev _saatchifinal.nc which is a combination of america/africa/asia I checked it with ncdump & ncks ....Henry

On 29/09/15 20:31, David LeBauer wrote:

List of combined files are here: https://gist.github.com/dlebauer/7cc778317e45bb36a74b

@hmb1 https://github.com/hmb1 did the following

• combined files with multiple variables into a subset of files o here is a list of files and variables https://gist.github.com/dlebauer/7cc778317e45bb36a74b o previously, each lookup of a value required the same find nearest neighbor app/controllers/workflows_controller.rb https://github.com/ebimodeling/ghgvc/blob/master/app/controllers/workflows_controller.rb
• need to create a test harness in Ruby

— Reply to this email directly or view it on GitHub https://github.com/ebimodeling/ghgvc/issues/5#issuecomment-144164728.

[dlebauer]

I've made the Saatchi et al data available on box at https://uofi.box.com/saatchi-ncdf

This includes a global map of aboveground biomass.

I've also removed refactoring code from the scope of this issue - that is covered in #70

@hmb1 did you combine the crops data listed under find /home/hmb/Crops/ -name *.nc?

[potterzot]

I think this should be closed, since it's essentially a replication of #59 and #64.