search

PecanProject / pecan

The Predictive Ecosystem Analyzer (PEcAn) is an integrated ecological bioinformatics toolbox.
www.pecanproject.org
Other
203 stars 235 forks source link

Web interface - ED2 soil / veg inputs (OGE/FAO) #525

Closed serbinsh closed 9 years ago

serbinsh commented 9 years ago

I am trying to sort out an issue with setting up ED2 runs on the web interface. When I get to the "03-inputs.php" page it looks like the OGE and FAO options are flipped, in that I am getting FAO for the veg and OGE for soil (see screenshot), but this should be the other way around, correct? That is how I have had it setup in my stand-alone ED2 runs in the past.

screen shot 2015-06-30 at 12 49 05 pm

serbinsh commented 9 years ago

I used the add.data.sh code to update the location of our site/ed_inputs and FAO/OGE:

#!/bin/bash

#DIRNAME=$( dirname $0 )
DIRNAME=/data/Model_Data
PDIR=/data/software/pecan

# folder to data, this is assumed to be installed at the same level
# as the pecan folder. The python code is to get the absolute path
# since the MAC does not have the GNU readlink -f option.
SITES=$( python -c 'import os,sys;print os.path.realpath(sys.argv[1])' "${DIRNAME}/sites" )
DALEC=$( python -c 'import os,sys;print os.path.realpath(sys.argv[1])' "${DIRNAME}/dalec_EnKF_pub/input_data" )
ED_INPUT=$( python -c 'import os,sys;print os.path.realpath(sys.argv[1])' "${DIRNAME}/ed_inputs" )
FAO_INPUT=$( python -c 'import os,sys;print os.path.realpath(sys.argv[1])' "${DIRNAME}/faoOLD" )
OGE2_INPUT=$( python -c 'import os,sys;print os.path.realpath(sys.argv[1])' "${DIRNAME}/oge2OLD" )

# load helper functions and set FQDN and PSQL
. ${PDIR}/scripts/add.util.sh

# ED inputs
addInputFile "${FQDN}" "294" "" "${ED_INPUT}/glu"
addInputFile "${FQDN}" "295" "" "${ED_INPUT}"
addInputFile "${FQDN}" "296" "FAO_" "${FAO_INPUT}"
addInputFile "${FQDN}" "297" "OGE2_" "${OGE2_INPUT}"

I made some modifications but did not change the IDs for OGE and FAO

serbinsh commented 9 years ago

I get these errors during the run:

+------------------------------------------------------------+
  => Generating the land/sea mask.
/data/Model_Data/faoOLD/FAO_HEADER
 -> Getting file: /data/Model_Data/faoOLD/FAO_30N120W.h5...
HDF5-DIAG: Error detected in HDF5 (1.8.14) MPI-process 0:
  #000: H5D.c line 358 in H5Dopen2(): not found
    major: Dataset
    minor: Object not found
  #001: H5Gloc.c line 430 in H5G_loc_find(): can't find object
    major: Symbol table
    minor: Object not found
  #002: H5Gtraverse.c line 861 in H5G_traverse(): internal path traversal failed
    major: Symbol table
    minor: Object not found
  #003: H5Gtraverse.c line 641 in H5G_traverse_real(): traversal operator failed
    major: Symbol table
    minor: Callback failed
  #004: H5Gloc.c line 385 in H5G_loc_find_cb(): object 'oge2' doesn't exist
    major: Symbol table
    minor: Object not found
HDF5-DIAG: Error detected in HDF5 (1.8.14) MPI-process 0:
  #000: H5D.c line 460 in H5Dget_space(): not a dataset
    major: Invalid arguments to routine
    minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.8.14) MPI-process 0:
  #000: H5D.c line 534 in H5Dget_type(): not a dataset
    major: Invalid arguments to routine
    minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.8.14) MPI-process 0:
  #000: H5Dio.c line 140 in H5Dread(): not a dataset
    major: Invalid arguments to routine
    minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.8.14) MPI-process 0:
  #000: H5S.c line 392 in H5Sclose(): not a dataspace
    major: Invalid arguments to routine
    minor: Inappropriate type
HDF5-DIAG: Error detected in HDF5 (1.8.14) MPI-process 0:
  #000: H5D.c line 415 in H5Dclose(): not a dataset
    major: Invalid arguments to routine
    minor: Inappropriate type
 -> Getting file: /data/Model_Data/faoOLD/FAO_30N090W.h5...
robkooper commented 9 years ago

This is what I have in the database:

 id  |          name          
-----+------------------------
 297 | ED soil database
 296 | ED vegetation database
(2 rows)

FAO is linked to 296 OGE is linked to 297

is this correct?

serbinsh commented 9 years ago

I think that is backwards. At least my working knowledge is: FAO - soil OGE - vegetation.

If you run ED2 with VEG_DATABASE set to FAO it gives those warnings above

serbinsh commented 9 years ago

@mdietze Can you confirm that I am not getting this backwards? I checked another PEcAn run of mine (manually setting the locations in the xml) and it ran fine, with those flipped

serbinsh commented 9 years ago

Weird, in my formats table through the web interface I see this:

screen shot 2015-06-30 at 1 10 48 pm

serbinsh commented 9 years ago

If I manually flip at the config edit stage I don't get that error:

   !---------------------------------------------------------------------------------------!
   !  Input databases                                                                      !
   !  VEG_DATABASE     -- vegetation database, used only to determine the land/water mask. !
   !                      Fill with the path and the prefix.                               !
   !  SOIL_DATABASE    -- soil database, used to determine the soil type.  Fill with the   !
   !                      path and the prefix.                                             !
   !  LU_DATABASE      -- land-use change disturbance rates database, used only when       !
   !                      IANTH_DISTURB is set to 1.  Fill with the path and the prefix.   !
   !  PLANTATION_FILE  -- plantation fraction file.  In case you don't have such a file or !
   !                      you do not want to use it, you must leave this variable empty:   !
   !                      (NL%PLANTATION_FILE = ''                                         !
   !  THSUMS_DATABASE  -- input directory with dataset to initialise chilling degrees and  !
   !                      growing degree days, which is used to drive the cold-deciduous   !
   !                      phenology (you must always provide this, even when your PFTs are !
   !                      not cold deciduous).                                             !
   !  ED_MET_DRIVER_DB -- File containing information for meteorological driver            !
   !                      instructions (the "header" file).                                !
   !  SOILSTATE_DB     -- Dataset in case you want to provide the initial conditions of    !
   !                      soil temperature and moisture.                                   !
   !  SOILDEPTH_DB     -- Dataset in case you want to read in soil depth information.      !
   !---------------------------------------------------------------------------------------!
   NL%VEG_DATABASE      = '/data/Model_Data/oge2OLD/OGE2_'
   NL%SOIL_DATABASE     = '/data/Model_Data/faoOLD/FAO_'
   NL%LU_DATABASE       = '/data/Model_Data/ed_inputs/glu/'
   NL%PLANTATION_FILE   = ''
   NL%THSUMS_DATABASE   = '/data/Model_Data/ed_inputs/'
   NL%ED_MET_DRIVER_DB = '/data/Model_Data/sites/US-WCr/ED_MET_DRIVER_HEADER'
   NL%SOILSTATE_DB=''
   NL%SOILDEPTH_DB=''
   !---------------------------------------------------------------------------------------!
+--- Parallel info: -------------------------------------+
+  - Machnum  =      0
+  - Machsize =      1
+--------------------------------------------------------+
Reading namelist information
Copying namelist
+------------------------------------------------------------+
|           Ecosystem Demography Model, version 2.2
+------------------------------------------------------------+
|  Input namelist filename is ED2IN
|
|  Single process execution on INITIAL run.
+------------------------------------------------------------+
  => Generating the land/sea mask.
/data/Model_Data/oge2OLD/OGE2_HEADER
 -> Getting file: /data/Model_Data/oge2OLD/OGE2_30N120W.h5...
 -> Getting file: /data/Model_Data/oge2OLD/OGE2_30N090W.h5...
 + Work allocation, node     1;
 + Polygon array allocation, node     1;
 + Memory successfully allocated on none     1;
 [+] Load_Ed_Ecosystem_Params...
 [+] Checking for XML config...

 ============================================
 Counting PFT(s) in XML: /data/Model_Output/pecan.output//PEcAn_2000000007/run/2000000006/config.xml
 Found            1  PFT tags
 MAXPFT =           11
 Opening ED2 XML Config file/data/Model_Output/pecan.output//PEcAn_2000000007/run/2000000006/config.xml
 EXTERN READ FROM FILE ::           0
 MISC READ FROM FILE ::           0
 ED_MISC READ FROM FILE ::           1
 PFT's READ FROM FILE ::           1
robkooper commented 9 years ago

Following statements should fix this.

@mdietze @dlebauer can you run this on your database.

update dbfiles set container_id=297 where file_name='FAO_';
update dbfiles set container_id=296 where file_name='OGE2_';
dlebauer commented 9 years ago

I ran these, but zero rows were affected.

On Tue, Jun 30, 2015 at 1:23 PM, Rob Kooper notifications@github.com wrote:

Closed #525 https://github.com/PecanProject/pecan/issues/525.

— Reply to this email directly or view it on GitHub https://github.com/PecanProject/pecan/issues/525#event-344543163.

serbinsh commented 9 years ago

@robkooper @mdietze @dlebauer Did this work?

dlebauer commented 9 years ago

I ran the code ... there were no updates. I don't have an up to date version of pecan running on ebi-forecast to test

serbinsh commented 9 years ago

@robkooper after all the updates, OGE and FAO are still flipped on my web interface?? I have reloaded the DB. DO I still need to run the code above?

serbinsh commented 9 years ago

Nevermind, works OK. I ran that code to correct the issue. Now works fine