Closed rmclaren closed 6 months ago
should we go ahead and uncomment test_iodaconv_prepbufr_ncep_aircftprofiles2ioda
?
should I push this up, we will get the CI testing output again then.
I'll go ahead and do this, @rmclaren please change as you see fit
@BenjaminRuston Fixed something. Could you download this branch and see if it fixes the problem on all the different machines?
nevermind
So recreated the whole environment on a VM with Spack-Stack (Ubuntu 20.04) and the test still passes. My configuration looks as follows:
source /home/rmclaren/spack-stack/setup.sh
spack env activate /home/rmclaren/spack-stack/envs/unified-env.mylinux
source /etc/profile.d/modules.sh
module use ${SPACK_STACK_DIR}/envs/unified-env.mylinux/install/modulefiles/Core
module load stack-gcc/9.4.0
module load stack-python/3.10.8
module load stack-mpich/4.1.2
module load ecflow
module load ewok-env
module load jedi-fv3-env
module load soca-env
The key thing here is I'm not loading anything that was not built as part of the stack.... Could anyone try in a failing environment?
@rmclaren I will give it a shot on my Mac. I have several versions of the spack stack installed. I also can try on orion with intel.
@rmclaren I am seeing the same failure on ubuntu8 and redhat using the gcc version 12 compiler and spack-stack 1.5.1. Same output on both.
What does your module config look like for ubuntu. I set up a fresh vm of 20.04, loaded the spack stack, but have absolutely no problems running that test... (gcc 9.4.0 in my case)
@PatNichols what happens if you cd <your ioda-converter dir>/build/test
and run the following valgrind --tool=memcheck --gen-suppressions=all --leak-check=full --leak-resolution=med --track-origins=yes "../bin/bufr2ioda.x testinput/bufr_ncep_prepbufr_aircft.yaml"
? You might need to install valgrind first...
valgrind: ./bin.bufr2ioda.x: No such file or directory
ec2-user bld2$ valgrind --leak-check=full -s ./bin/bufr2ioda.x bufr_ncep_prepbufr_aircft.yaml
==34049== Memcheck, a memory error detector
==34049== Copyright (C) 2002-2022, and GNU GPL'd, by Julian Seward et al.
==34049== Using Valgrind-3.21.0 and LibVEX; rerun with -h for copyright info
==34049== Command: ./bin/bufr2ioda.x bufr_ncep_prepbufraircft.yaml
==34049==
==34049== Warning: set address range perms: large range [0x1d1ad040, 0x328ffa40) (undefined)
==34049== Warning: set address range perms: large range [0x40795040, 0x575cb040) (undefined)
vex amd64->IR: unhandled instruction bytes: 0xC4 0xE1 0xF9 0x90 0x10 0xB8 0x2 0x0 0x0 0x0
vex amd64->IR: REX=0 REX.W=1 REX.R=0 REX.X=0 REX.B=0
vex amd64->IR: VEX=1 VEX.L=0 VEX.nVVVV=0x0 ESC=0F
vex amd64->IR: PFX.66=1 PFX.F2=0 PFX.F3=0
==34049== valgrind: Unrecognised instruction at address 0x82fc9d8.
==34049== at 0x82FC9D8: wrdlen (wrdlen.F:24)
==34049== by 0x82FBE4A: openbf_ (openbf.f:202)
==34049== by 0x82FF87E: openbf_f (bufr_c2f_interface.F90:127)
==34049== by 0x40DFA67: Ingester::bufr::NcepDataProvider::open() (NcepDataProvider.cpp:25)
==34049== by 0x40E3010: Ingester::bufr::File::File(std::cxx11::basic_string<char, std::char_traits
@rmclaren Not sure how well valgrind handles fortran though.
@PatNichols Thanks!
@rmclaren Any progress on this. After some testing it looks like valgrinds output is misleading. The common block is triggering an error when none exists.
Sorry, trying to finish something up and then circle back to this.
Little of an update. I finally got them to give me an account on ORION and the branch feature/query_acprofile_prob dooes not reproduce the problem on those machines (the develop branch will have a different issue because something got commented out in the YAML file just FYI).
@rmclaren I checked the output of ioda_validate and at least for pressure there are problems. This may be problem with the input file not the code itself.
.... Reading YAML from /build_container/ioda/share/test/testinput/validation/ObsSpace.yaml Processing data file: /jcsda/ioda-bundle/iodaconv/test/testoutput/gdas.t12z.acft_profiles.prepbufr.nc
.... Variable QualityMarker/pressure Warning: Variable 'QualityMarker/pressure' does not have match any of the recommended dimensions. Variable dimensions: [ Location dim_2 PressureEvent ]. Recommended dimensions: [ Location ] [ Location Level ] [ Location Level PressureEvent ] Variable QualityMarker/airTemperature Warning: Variable 'QualityMarker/airTemperature' does not have match any of the recommended dimensions. Variable dimensions: [ Location dim_2 TemperatureEvent ]. Recommended dimensions: [ Location ] [ Location Level ] [ Location Level TemperatureEvent ] [ Location AmdarSequence ] [ Location dim_2 ] Variable QualityMarker/specificHumidity Warning: Variable 'QualityMarker/specificHumidity' does not have match any of the recommended dimensions. Variable dimensions: [ Location dim_2 HumidityEvent ]. Recommended dimensions: [ Location ] [ Location Level ] [ Location Level HumidityEvent ] Variable QualityMarker/windEastward Warning: Variable 'QualityMarker/windEastward' does not have match any of the recommended dimensions. Variable dimensions: [ Location dim_2 WindEvent ]. Recommended dimensions: [ Location ] [ Location Level ] [ Location Level WindEvent ] Variable QualityMarker/height Warning: Variable 'QualityMarker/height' does not have match any of the recommended dimensions. Variable dimensions: [ Location dim_2 HeightEvent ]. Recommended dimensions: [ Location ] [ Location Level ] [ Location Level HeightEvent ] Variable QualityMarker/windNorthward Warning: Variable 'QualityMarker/windNorthward' does not have match any of the recommended dimensions. Variable dimensions: [ Location dim_2 WindEvent ]. Recommended dimensions: [ Location ] [ Location Level ] [ Location Level WindEvent ] Variable ObsError/pressure Warning: Variable 'ObsError/pressure' does not have match any of the recommended dimensions. Variable dimensions: [ Location dim_2 ]. Recommended dimensions: [ Location ] [ Location Level ] [ Location Level PressureEvent ] Variable ObsError/airTemperature Variable ObsError/relativeHumidity ...
@PatNichols Your comment is unrelated to the issue in this thread. It just means that the input YAML file isn't following those conventions...
I'm going to close this Pull Review and open a new one for feature/query_acprofile_prob_2 in order to merge with a cleaner history. Please see https://github.com/JCSDA-internal/ioda-converters/pull/1467.
Description
Find fix for acprofile test problem