bug: the AIRS converter doesn't work with the latest HDF-EOS2-3.0 library

nancycollins commented 8 months ago

:bug: This isn't our bug, it's an apparent bug in the HDF-EOS library. i've sent an email to what seems to be HDF-EOS support to see if it's a known bug. this issue is to track and report if i hear anything back from them. there's not much we can do on the DART side other than patch their library which is already done.

Describe the bug

use the hdf-eos and rttov libraries that helen built on: /glade/campaign/cisl/dares/libraries
build the observations/obs_converters/AIRS/work/convert_airs_L2 with quickbuild.sh
run it. the filenames are truncated 1 space too soon. e.g. a filename of "bob" becomes "bo" and it fails.

Error Message

the converter fails because the strings aren't being passed from fortran to c correctly. the initial symptom is a 'file not found' error which is correct since the code is looking for the wrong filename.

Which model(s) are you working with?

AIRS obs converter

Have you modified the DART code?

I modified include/fortran_api.h in the hdf-eos distribution, in particular line 81: if (len == maxlen) while(len > 0 && i[len] == ' ') --len;

the corrected version: if (len == maxlen) while(len > 0 && i[len-1] == ' ') --len;

Build information

see the README in /glade/campaign/cisl/dares/libraries for help compiling

i built it on cheyenne with intel for debugging; it can only be built on derecho if you turn off warnings.

hkershaw-brown commented 8 months ago

@nancycollins does this create obs on Cheyenne? I built the patched version of hdf-eos on Derecho. /glade/campaign/cisl/dares/libraries/hdf-eos.patch/

The converter runs, but there are 0 obs in the obs_sequence file

hkershaw@derecho8:/glade/derecho/scratch/hkershaw/DART/Bugs/nick_airs$ ./convert_airs_L2 

 --------------------------------------
 Starting ... at YYYY MM DD HH MM SS = 
                 2023 12  7 12 33 35
 Program convert_airs_L2
 --------------------------------------

  set_nml_output Echo NML values to log file only
   obs used =            1
  print_obs_seq_summary observation sequence is empty
  write_obs_seq  opening formatted observation sequence file "obs_seq.test"
  location_mod: using code with optimized cutoffs
  location_mod: Ignoring vertical separation when computing distances; horizonta
 l distances only
  write_obs_seq  closed observation sequence file "obs_seq.test"
  convert_airs_L2.f90 Finished successfully.

 --------------------------------------
 Finished ... at YYYY MM DD HH MM SS = 
                 2023 12  7 12 33 35
 --------------------------------------

hkershaw@derecho8:/glade/derecho/scratch/hkershaw/DART/Bugs/nick_airs$ cat obs_seq.test 
 obs_sequence
obs_type_definitions
           0
  num_copies:            1  num_qc:            1
  num_obs:            0  max_num_obs:            0
observation                                                     
AIRS QC                                                         
  first:           -1  last:           -1

nancycollins commented 8 months ago

edit your input.nml to go from -90 to 90 latitude and set the thin values both to 0 and try again? i had a couple files where all the obs were outside of the default -60 to 60 latitude range.

hkershaw-brown commented 8 months ago

ok thanks I had -90 90, but had the default thinning (1, 1)

nancycollins commented 8 months ago

also, bug: obs_num printing out num of obs converted. should print obs_num - 1 at end.

nancycollins commented 8 months ago

the input.nml defaults need to be something other than thin out every obs. i'd like to update the code to print the right number of obs and make the input.nml defaults more sane. can i do that in this same issue or should i open another one?

hkershaw-brown commented 8 months ago

new bug new issue!

hkershaw-brown commented 8 months ago

fix for this is update AIRS documentation:

patched version available on Derecho for the various compilers

/glade/campaign/cisl/dares/libraries/hdf-eos_* hdf-eos_cray hdf-eos_gfortran hdf-eos_intel hdf-eos_nvhpc

recomend users use intel. (ifx or ifort can be linked to hdf-eos_intel)

example mkmf.template for intel and gfotran /glade/campaign/cisl/dares/libraries/hdf-eos_intel/mkmf.template /glade/campaign/cisl/dares/libraries/hdf-eos_gfortran/mkmf.template

Notes on patch to go into the dart docs.

Fix up dependency info in docs see #534 #531

nancycollins commented 6 months ago

2 months later i got a reply from the HDF-EOS support folks. they have been having trouble with their ticket system.

i sent them the error and fix. i will update this issue if/when i get something back.

NCAR / DART