search

NOAA-EMC / NCEPLIBS-w3emc

This library contains Fortran 77 decoder/encoder routines for GRIB edition 1.
Other
2 stars 8 forks source link

w3emc upgrade on WCOSS2? #234

Open ShelleyMelchior-NOAA opened 3 months ago

ShelleyMelchior-NOAA commented 3 months ago

Hi NCEPlibs team! I am just looking for a bit of guidance on steps forward as they relate to the next upgrade of EVS (v2).

The current operations version of EVS (v1.0.9) uses w3emc/2.9.2 and ip/3.3.3. We could not use ip/4.0.0 b/c of a dependency on ip/3.3.3 to make sure makgds worked properly. Once w3emc is upgraded on WCOSS2 to be w3emc/2.9.3, EVS can use ip/4.0.0.

EVSv2 development is underway and we want to know when w3emc will be upgraded on WCOSS2. We are also curious which version of w3emc will be installed since now I see there are releases up to w3emc.v2.12.0!

Any guidance you can provide is appreciated. @Hang-Lei-NOAA @edwardhartnett @AliciaBentley-NOAA @PerryShafran-NOAA @aerorahul

Hang-Lei-NOAA commented 3 months ago

Hi, Shelley,

We will start to install several newer versions of w3emc on acorn for you and others to test. This may include: w3emc/2.10.0 (This may be safe to your usage, close/comparable to 2.9.3, some other groups have already tested it on acorn it is under /lfs/h1/emc /nceplibs/noscrub/hpc-stack/libs/hpc-stack/modulefiles/compiler/intel/19.1.3.304 You can test this one now)

the following two versions will need more testing. w3emc/2.12.0 W3emc/2.13.0

Thanks, Hang Lei

On Tue, Jun 25, 2024 at 1:50 PM Shelley Melchior @.***> wrote:

Hi NCEPlibs team! I am just looking for a bit of guidance on steps forward as they relate to the next upgrade of EVS (v2).

The current operations version of EVS (v1.0.9) uses w3emc/2.9.2 and ip/3.3.3. We could not use ip/4.0.0 b/c of a dependency on ip/3.3.3 to make sure makgds worked properly. Once w3emc is upgraded on WCOSS2 to be w3emc/2.9.3, EVS can use ip/4.0.0.

EVSv2 development is underway and we want to know when w3emc will be upgraded on WCOSS2. We are also curious which version of w3emc will be installed since now I see there are releases up to w3emc.v2.12.0!

Any guidance you can provide is appreciated. @Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA @edwardhartnett https://github.com/edwardhartnett @AliciaBentley-NOAA https://github.com/AliciaBentley-NOAA @PerryShafran-NOAA https://github.com/PerryShafran-NOAA @aerorahul https://github.com/aerorahul

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/NCEPLIBS-w3emc/issues/234, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFC3FPTBKH7HZIUQS6LZJGUV5AVCNFSM6AAAAABJ4KMXMKVHI2DSMVQWIX3LMV43ASLTON2WKOZSGM3TGMRYHE3DCMA . You are receiving this because you were mentioned.Message ID: @.***>

edwardhartnett commented 3 months ago

@Hang-Lei-NOAA can we just jump to the latest release, and skip intermediate releases?

Hang-Lei-NOAA commented 3 months ago

Yes, we can jump to 2.13.0 since 2.12.0 is very close to that. But it’s better to keep 2.10.0 for less risk, since some codes after it have been decreased. Users have tested 2.10.0. We actually not exactly sure if this decrease will affect some rare functions in models. 2.10.0 represents a big stage for the transformation of w3emc.

On Wed, Jun 26, 2024 at 6:24 AM Edward Hartnett @.***> wrote:

@Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA can we just jump to the latest release, and skip intermediate releases?

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/NCEPLIBS-w3emc/issues/234#issuecomment-2191346578, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFCAPOUMCVSID7LEJSDZJKJGTAVCNFSM6AAAAABJ4KMXMKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJRGM2DMNJXHA . You are receiving this because you were mentioned.Message ID: @.***>

ShelleyMelchior-NOAA commented 3 months ago

Thanks @Hang-Lei-NOAA and @edwardhartnett . We can do a baseline test of w3emc/2.10.0 on acorn by trying to compile the code that uses makgds, but will it ever be promoted to a prod installation on dogwood or cactus? EVS runs in ops so we cannot use libraries on acorn for operational runs.

ShelleyMelchior-NOAA commented 2 months ago

Hi @Hang-Lei-NOAA and @edwardhartnett , I wanted to repeat my question in case it was overlooked with holiday leave.

Is it expected that w3emc/2.10.0, or any other future version, will be promoted to a prod installation on dogwood or cactus?

Hang-Lei-NOAA commented 2 months ago

@ShelleyMelchior-NOAA Have you tested with these versions that I installed on acorn it is under /lfs/h1/emc /nceplibs/noscrub/hpc-stack/libs/hpc-stack/modulefiles/compiler/intel/19.1.3.304

Once you confirmed working, we can start the delivery.

ShelleyMelchior-NOAA commented 2 months ago

Ok, thanks @Hang-Lei-NOAA . That answers my question. We will test the installation on acorn and get back to you.

I am uncertain how to access acorn. Before I task a team member who can access acorn to test the w3emc/2.10.0 installation, can you confirm that all other modules are installed on acorn that the EVS sorc code needs for successful compilation: w3emc, ip, bacio, sp, g2, jasper, libpng, zlib? If so, are they installed at the same location as w3emc/2.10.0?

Hang-Lei-NOAA commented 2 months ago

@Shelley Melchior - NOAA Affiliate @.***> I will share a testing installation to you for personal test.

On Wed, Jul 10, 2024 at 5:40 PM Shelley Melchior @.***> wrote:

Ok, thanks @Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA . That answers my question. We will test the installation on acorn and get back to you.

I am uncertain how to access acorn. Before I task a team member who can access acorn to test the w3emc/2.10.0 installation, can you confirm that all other modules are installed on acorn that the EVS sorc code needs for successful compilation: w3emc, bacio, sp, g2, jasper, libpng, zlib? If so, are they installed at the same location as w3emc/2.10.0?

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/NCEPLIBS-w3emc/issues/234#issuecomment-2221566117, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFADFBIL32ISQGI2TQ3ZLWS4ZAVCNFSM6AAAAABJ4KMXMKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRRGU3DMMJRG4 . You are receiving this because you were mentioned.Message ID: @.***>

ShelleyMelchior-NOAA commented 2 months ago

Thank you, @Hang-Lei-NOAA ! I was able to test the compiling of the EVS source code with the test installation for w3emc/2.10.0 and w3emc/2.12.0. I was also pointing to the latest production installation for ip/4.0.0. Good news! Both versions of w3emc compile with no errors and generate the executable files. The EVS team is endorsing the delivery of the w3emc upgrade to NCO.

Hang-Lei-NOAA commented 2 months ago

Awesome, thanks for the update! I will prepare the delivery soon. Hopefully, will send out next Monday in waiting for some other libs. Before that you can still testing this one.

Thanks Hang

On Thu, Jul 11, 2024 at 2:43 PM Shelley Melchior @.***> wrote:

Thank you, @Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA ! I was able to test the compiling of the EVS source code with the test installation for w3emc/2.10.0 and w3emc/2.12.0. I was also pointing to the latest production installation for ip/4.0.0. Good news! Both versions of w3emc compile with no errors and generate the executable files. The EVS team is endorsing the delivery of the w3emc upgrade to NCO.

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/NCEPLIBS-w3emc/issues/234#issuecomment-2223650847, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFCC5XHACSLAWI3OEILZL3G3NAVCNFSM6AAAAABJ4KMXMKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRTGY2TAOBUG4 . You are receiving this because you were mentioned.Message ID: @.***>

ShelleyMelchior-NOAA commented 2 months ago

Fantastic! Can you keep me apprised of the progress of the release and installation in prod? We will want to update the EVS code in development to point to the upgraded w3emc library once it is installed in operations.

Also, for your awareness, we are targeting this for use in EVSv2, which will be delivered to NCO in May 2025.

Hang-Lei-NOAA commented 2 months ago

Yes, I will let you know.

On Thu, Jul 11, 2024 at 3:04 PM Shelley Melchior @.***> wrote:

Fantastic! Can you keep me apprised of the progress of the release and installation in prod? We will want to update the EVS code in development to point to the upgraded w3emc library once it is installed in operations.

Also, for your awareness, we are targeting this for use in EVSv2, which will be delivered to NCO in May 2025.

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/NCEPLIBS-w3emc/issues/234#issuecomment-2223685704, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFFW2TZK2ELZF3WZHYDZL3JMFAVCNFSM6AAAAABJ4KMXMKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRTGY4DKNZQGQ . You are receiving this because you were mentioned.Message ID: @.***>

Hang-Lei-NOAA commented 2 months ago

Hi Shelley

NCO has quickly installed the latest library on cactus, please test it as: @.:~> module use /apps/ops/para/libs/modulefiles/compiler/intel/19.1.3.304 @.:~> module load g2/3.5.1 @.***:~> module show g2/3.5.1

/apps/ops/para/libs/modulefiles/compiler/intel/19.1.3.304/g2/3.5.1.lua:

help([[]]) conflict("g2") setenv("g2_ROOT","/apps/ops/para/libs/intel/19.1.3.304/g2/3.5.1") setenv("g2_VERSION","3.5.1") setenv("G2_INC4","/apps/ops/para/libs/intel/19.1.3.304/g2/3.5.1/include_4") setenv("G2_INCd","/apps/ops/para/libs/intel/19.1.3.304/g2/3.5.1/include_d") setenv("G2_LIB4","/apps/ops/para/libs/intel/19.1.3.304/g2/3.5.1/lib64/libg2_4.a") setenv("G2_LIBd","/apps/ops/para/libs/intel/19.1.3.304/g2/3.5.1/lib64/libg2_d.a") whatis("Name: g2") whatis("Version: 3.5.1") whatis("Category: library") whatis("Description: g2 library")

@.:~> ls /apps/ops/para/libs/intel/19.1.3.304/g2/3.5.1/include_4 drstemplates.mod g2grids.mod g2logging.mod grib_mod.mod gridtemplates.mod intmath.mod params_ecmwf.mod params.mod pdstemplates.mod re_alloc.mod @.:~> module show w3emc/2.12.0

/apps/ops/para/libs/modulefiles/compiler/intel/19.1.3.304/w3emc/2.12.0.lua:

help([[]]) conflict("w3emc") setenv("w3emc_ROOT","/apps/ops/para/libs/intel/19.1.3.304/w3emc/2.12.0") setenv("w3emc_VERSION","2.12.0") setenv("W3EMC_INC4","/apps/ops/para/libs/intel/19.1.3.304/w3emc/2.12.0/include_4") setenv("W3EMC_INC8","/apps/ops/para/libs/intel/19.1.3.304/w3emc/2.12.0/include_8") setenv("W3EMC_INCd","/apps/ops/para/libs/intel/19.1.3.304/w3emc/2.12.0/include_d") setenv("W3EMC_LIB4","/apps/ops/para/libs/intel/19.1.3.304/w3emc/2.12.0/lib64/libw3emc_4.a") setenv("W3EMC_LIB8","/apps/ops/para/libs/intel/19.1.3.304/w3emc/2.12.0/lib64/libw3emc_8.a") setenv("W3EMC_LIBd","/apps/ops/para/libs/intel/19.1.3.304/w3emc/2.12.0/lib64/libw3emc_d.a") whatis("Name: w3emc") whatis("Version: 2.12.0") whatis("Category: library") whatis("Description: w3emc library")

@.:~> ls /apps/ops/para/libs/intel/19.1.3.304/w3emc/2.12.0/include_4/ mersenne_twister.mod @.:~> module show g2tmpl g2tmpl g2tmpl/1.10.0 g2tmpl/1.10.2 g2tmpl/1.12.0 g2tmpl/1.13.0 g2tmpl/1.9.1 @.:~> module show g2tmpl g2tmpl g2tmpl/1.10.0 g2tmpl/1.10.2 g2tmpl/1.12.0 g2tmpl/1.13.0 g2tmpl/1.9.1 @.:~> module show g2tmpl g2tmpl g2tmpl/1.10.0 g2tmpl/1.10.2 g2tmpl/1.12.0 g2tmpl/1.13.0 g2tmpl/1.9.1 @.***:~> module show g2tmpl/1.13.0

/apps/ops/para/libs/modulefiles/compiler/intel/19.1.3.304/g2tmpl/1.13.0.lua:

help([[]]) conflict("g2tmpl") setenv("g2tmpl_ROOT","/apps/ops/para/libs/intel/19.1.3.304/g2tmpl/1.13.0") setenv("g2tmpl_VERSION","1.13.0") setenv("G2TMPL_INC","/apps/ops/para/libs/intel/19.1.3.304/g2tmpl/1.13.0/include") setenv("G2TMPL_LIB","/apps/ops/para/libs/intel/19.1.3.304/g2tmpl/1.13.0/lib64/libg2tmpl.a") whatis("Name: g2tmpl") whatis("Version: 1.13.0") whatis("Category: library") whatis("Description: g2tmpl library")

@.***:~> ls /apps/ops/para/libs/intel/19.1.3.304/g2tmpl/1.13.0/include grib2_all_tables_module.mod

On Thu, Jul 11, 2024 at 3:08 PM Hang Lei - NOAA Affiliate @.***> wrote:

Yes, I will let you know.

On Thu, Jul 11, 2024 at 3:04 PM Shelley Melchior @.***> wrote:

Fantastic! Can you keep me apprised of the progress of the release and installation in prod? We will want to update the EVS code in development to point to the upgraded w3emc library once it is installed in operations.

Also, for your awareness, we are targeting this for use in EVSv2, which will be delivered to NCO in May 2025.

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/NCEPLIBS-w3emc/issues/234#issuecomment-2223685704, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFFW2TZK2ELZF3WZHYDZL3JMFAVCNFSM6AAAAABJ4KMXMKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRTGY4DKNZQGQ . You are receiving this because you were mentioned.Message ID: @.***>

ShelleyMelchior-NOAA commented 2 months ago

Thank you for the update, @Hang-Lei-NOAA ! I will test the para install and report back.

ShelleyMelchior-NOAA commented 2 months ago

Hi @Hang-Lei-NOAA ! When I previously tested the update for w3emc, I did not realize there was an update to g2. My tests were using the prod installation of g2/3.4.5. Compilation errors occur when I use v3.5.1 for g2.

✔️ EVS compiles correctly when I use the cactus installation at /apps/ops/para/libs/modulefiles/compiler/intel/19.1.3.304 for w3emc/2.12.0, ip/4.0.0, and the prod installation for g2/3.4.5.

❌ EVS does not compile correctly when I use the cactus installation at /apps/ops/para/libs/modulefiles/compiler/intel/19.1.3.304 for w3emc/2.12.0, ip/4.0.0, and g2/3.5.1.

Do we have to use the updated g2 to use updated w3emc and ip?

Provided is the compilation error that I see:

ftn -g -traceback -O3 -auto -convert big_endian -I /apps/ops/para/libs/intel/19.1.3.304/g2/3.5.1/include_4 -o sref_precip.x sref_precip.f /apps/ops/prod/libs/intel/19.1.3.304/ip/4.0.0/lib64/libip_4.a /apps/ops/para/libs/intel/19.1.3.304/w3emc/2.12.0/lib64/libw3emc_4.a /apps/ops/prod/libs/intel/19.1.3.304/bacio/2.4.1/lib/libbacio_4.a  /apps/ops/para/libs/intel/19.1.3.304/sp/2.4.0/lib64/libsp_4.a /apps/ops/para/libs/intel/19.1.3.304/g2/3.5.1/lib64/libg2_4.a /apps/spack/jasper/2.0.25/intel/19.1.3.304/sjib74krrorkyczqpqah4tvewmlnqdx4/lib64/libjasper.a /apps/spack/libpng/1.6.37/intel/19.1.3.304/4ohkronuhlyherusoszzrmur5ewvlwzh/lib/libpng.a /apps/spack/zlib/1.2.11/intel/19.1.3.304/hjotqkckeoyt6j6tibalwzrlfljcjtdh/lib/libz.a
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: /apps/spack/jasper/2.0.25/intel/19.1.3.304/sjib74krrorkyczqpqah4tvewmlnqdx4/lib64/libjasper.a(jpg_dec.c.o): in function `jpg_decode':
jpg_dec.c:(.text+0x237): undefined reference to `jpeg_std_error'
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: jpg_dec.c:(.text+0x251): undefined reference to `jpeg_CreateDecompress'
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: jpg_dec.c:(.text+0x261): undefined reference to `jpeg_stdio_src'
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: jpg_dec.c:(.text+0x273): undefined reference to `jpeg_read_header'
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: jpg_dec.c:(.text+0x323): undefined reference to `jpeg_start_decompress'
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: jpg_dec.c:(.text+0x48f): undefined reference to `jpeg_read_scanlines'
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: jpg_dec.c:(.text+0x4dc): undefined reference to `jpeg_finish_decompress'
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: jpg_dec.c:(.text+0x4e9): undefined reference to `jpeg_destroy_decompress'
Hang-Lei-NOAA commented 2 months ago

No , we don’t require to use it. But it should be fine. if you want to use it. You need to load the JPEG library and add it into your building script, or makefile.

On Mon, Jul 22, 2024 at 12:48 PM Shelley Melchior @.***> wrote:

Hi @Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA ! When I previously tested the update for w3emc, I did not realize there was an update to g2. My tests were using the prod installation of g2/3.4.5. Compilation errors occur when I use v3.5.1 for g2.

✔️ EVS compiles correctly when I use the cactus installation at /apps/ops/para/libs/modulefiles/compiler/intel/19.1.3.304 for w3emc/2.12.0, ip/4.0.0, and the prod installation for g2/3.4.5.

❌ EVS does not compile correctly when I use the cactus installation at /apps/ops/para/libs/modulefiles/compiler/intel/19.1.3.304 for w3emc/2.12.0, ip/4.0.0, and g2/3.5.1.

Do we have to use the updated g2 to use updated w3emc and ip?

Provided is the compilation error that I see:

ftn -g -traceback -O3 -auto -convert big_endian -I /apps/ops/para/libs/intel/19.1.3.304/g2/3.5.1/include_4 -o sref_precip.x sref_precip.f /apps/ops/prod/libs/intel/19.1.3.304/ip/4.0.0/lib64/libip_4.a /apps/ops/para/libs/intel/19.1.3.304/w3emc/2.12.0/lib64/libw3emc_4.a /apps/ops/prod/libs/intel/19.1.3.304/bacio/2.4.1/lib/libbacio_4.a /apps/ops/para/libs/intel/19.1.3.304/sp/2.4.0/lib64/libsp_4.a /apps/ops/para/libs/intel/19.1.3.304/g2/3.5.1/lib64/libg2_4.a /apps/spack/jasper/2.0.25/intel/19.1.3.304/sjib74krrorkyczqpqah4tvewmlnqdx4/lib64/libjasper.a /apps/spack/libpng/1.6.37/intel/19.1.3.304/4ohkronuhlyherusoszzrmur5ewvlwzh/lib/libpng.a /apps/spack/zlib/1.2.11/intel/19.1.3.304/hjotqkckeoyt6j6tibalwzrlfljcjtdh/lib/libz.a /usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: /apps/spack/jasper/2.0.25/intel/19.1.3.304/sjib74krrorkyczqpqah4tvewmlnqdx4/lib64/libjasper.a(jpg_dec.c.o): in function jpg_decode': jpg_dec.c:(.text+0x237): undefined reference tojpeg_std_error' /usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: jpg_dec.c:(.text+0x251): undefined reference to jpeg_CreateDecompress' /usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: jpg_dec.c:(.text+0x261): undefined reference tojpeg_stdio_src' /usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: jpg_dec.c:(.text+0x273): undefined reference to jpeg_read_header' /usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: jpg_dec.c:(.text+0x323): undefined reference tojpeg_start_decompress' /usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: jpg_dec.c:(.text+0x48f): undefined reference to jpeg_read_scanlines' /usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: jpg_dec.c:(.text+0x4dc): undefined reference tojpeg_finish_decompress' /usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: jpg_dec.c:(.text+0x4e9): undefined reference to `jpeg_destroy_decompress'

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/NCEPLIBS-w3emc/issues/234#issuecomment-2243396164, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFG56Z6IF3POEYCHHPDZNUZWTAVCNFSM6AAAAABJ4KMXMKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBTGM4TMMJWGQ . You are receiving this because you were mentioned.Message ID: @.***>

ShelleyMelchior-NOAA commented 2 months ago

Thanks @Hang-Lei-NOAA . We do load libjpeg/9c, but it is not in the makefile. I added it to the makefile for the impacted source code. Here is the make command:

ftn -g -traceback -O3 -auto -convert big_endian -I /apps/ops/para/libs/intel/19.1.3.304/g2/3.5.1/include_4 -o sref_precip.x sref_precip.f /apps/ops/prod/libs/intel/19.1.3.304/ip/4.0.0/lib64/libip_4.a /apps/ops/para/libs/intel/19.1.3.304/w3emc/2.12.0/lib64/libw3emc_4.a /apps/ops/prod/libs/intel/19.1.3.304/bacio/2.4.1/lib/libbacio_4.a  /apps/ops/para/libs/intel/19.1.3.304/sp/2.4.0/lib64/libsp_4.a /apps/ops/para/libs/intel/19.1.3.304/g2/3.5.1/lib64/libg2_4.a /apps/spack/jasper/2.0.25/intel/19.1.3.304/sjib74krrorkyczqpqah4tvewmlnqdx4/lib64/libjasper.a /apps/spack/libpng/1.6.37/intel/19.1.3.304/4ohkronuhlyherusoszzrmur5ewvlwzh/lib/libpng.a /apps/spack/libjpeg/9c/intel/19.1.3.304/jkr3isi257ktoouprwaxcn4twtye747z/lib/libjpeg.a /apps/spack/zlib/1.2.11/intel/19.1.3.304/hjotqkckeoyt6j6tibalwzrlfljcjtdh/lib/libz.a

And here are a few errors that are spawned:

/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: /apps/spack/libjpeg/9c/intel/19.1.3.304/jkr3isi257ktoouprwaxcn4twtye747z/lib/libjpeg.a(jcapimin.o): in function `jpeg_CreateCompress':
/tmp/root/spack-stage/spack-stage-libjpeg-9c-jkr3isi257ktoouprwaxcn4twtye747z/spack-src/jcapimin.c:68: undefined reference to `__cray_dset_detect'
/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: /apps/spack/libjpeg/9c/intel/19.1.3.304/jkr3isi257ktoouprwaxcn4twtye747z/lib/libjpeg.a(jcmaster.o): in function `validate_script':
/tmp/root/spack-stage/spack-stage-libjpeg-9c-jkr3isi257ktoouprwaxcn4twtye747z/spack-src/jcmaster.c:209: undefined reference to `__cray_sset_detect'
Hang-Lei-NOAA commented 2 months ago

Hi, Shelley

Could you please replace the library by using module use /lfs/h2/emc/eib/save/hang.lei/forgdit/nco_wcoss2/install2/modulefiles/compiler/intel/19.1.3.304 module load g2/3.5.1 module load w3emc/2.12.0 to see if these error still occur?

If it still doesn't work, then please just test g2/3.4.5 for your case.

Thanks, Hang

On Mon, Jul 22, 2024 at 1:38 PM Shelley Melchior @.***> wrote:

Thanks @Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA . We do load libjpeg/9c, but it is not in the makefile. I added it to the makefile for the impacted source code. Here is the make command:

ftn -g -traceback -O3 -auto -convert big_endian -I /apps/ops/para/libs/intel/19.1.3.304/g2/3.5.1/include_4 -o sref_precip.x sref_precip.f /apps/ops/prod/libs/intel/19.1.3.304/ip/4.0.0/lib64/libip_4.a /apps/ops/para/libs/intel/19.1.3.304/w3emc/2.12.0/lib64/libw3emc_4.a /apps/ops/prod/libs/intel/19.1.3.304/bacio/2.4.1/lib/libbacio_4.a /apps/ops/para/libs/intel/19.1.3.304/sp/2.4.0/lib64/libsp_4.a /apps/ops/para/libs/intel/19.1.3.304/g2/3.5.1/lib64/libg2_4.a /apps/spack/jasper/2.0.25/intel/19.1.3.304/sjib74krrorkyczqpqah4tvewmlnqdx4/lib64/libjasper.a /apps/spack/libpng/1.6.37/intel/19.1.3.304/4ohkronuhlyherusoszzrmur5ewvlwzh/lib/libpng.a /apps/spack/libjpeg/9c/intel/19.1.3.304/jkr3isi257ktoouprwaxcn4twtye747z/lib/libjpeg.a /apps/spack/zlib/1.2.11/intel/19.1.3.304/hjotqkckeoyt6j6tibalwzrlfljcjtdh/lib/libz.a

And here are a few errors that are spawned:

/usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: /apps/spack/libjpeg/9c/intel/19.1.3.304/jkr3isi257ktoouprwaxcn4twtye747z/lib/libjpeg.a(jcapimin.o): in function jpeg_CreateCompress': /tmp/root/spack-stage/spack-stage-libjpeg-9c-jkr3isi257ktoouprwaxcn4twtye747z/spack-src/jcapimin.c:68: undefined reference to__cray_dset_detect' /usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: /apps/spack/libjpeg/9c/intel/19.1.3.304/jkr3isi257ktoouprwaxcn4twtye747z/lib/libjpeg.a(jcmaster.o): in function validate_script': /tmp/root/spack-stage/spack-stage-libjpeg-9c-jkr3isi257ktoouprwaxcn4twtye747z/spack-src/jcmaster.c:209: undefined reference to__cray_sset_detect'

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/NCEPLIBS-w3emc/issues/234#issuecomment-2243478188, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFGNUDF6PY7ZTU6TIJTZNU7P3AVCNFSM6AAAAABJ4KMXMKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBTGQ3TQMJYHA . You are receiving this because you were mentioned.Message ID: @.***>

ShelleyMelchior-NOAA commented 2 months ago

Yes! I can try this. One question. I don't see w3emc in the provided installation path. Should I use the para installation for w3emc/2.12.0?

$ ls /lfs/h2/emc/eib/save/hang.lei/forgdit/nco_wcoss2/install2/modulefiles/compiler/intel/19.1.3.304
g2  g2tmpl  hpc-cray-mpich
Hang-Lei-NOAA commented 2 months ago

Hi, Shelley

Please try it again. I just moved it there for you.

Thanks, Hang

On Mon, Jul 22, 2024 at 2:06 PM Shelley Melchior @.***> wrote:

Yes! I can try this. One question. I don't see w3emc in the provided installation path. Should I use the para installation for w3emc/2.12.0?

$ ls /lfs/h2/emc/eib/save/hang.lei/forgdit/nco_wcoss2/install2/modulefiles/compiler/intel/19.1.3.304 g2 g2tmpl hpc-cray-mpich

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/NCEPLIBS-w3emc/issues/234#issuecomment-2243526401, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFFP34VJ3HRSIQ6N35DZNVC23AVCNFSM6AAAAABJ4KMXMKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBTGUZDMNBQGE . You are receiving this because you were mentioned.Message ID: @.***>

ShelleyMelchior-NOAA commented 2 months ago

Thanks Hang! I tried to recompile using your installation. Here is the make command:

ftn -g -traceback -O3 -auto -convert big_endian -I /lfs/h2/emc/eib/save/hang.lei/forgdit/nco_wcoss2/install2/intel-19.1.3.304/g2/3.5.1/include_4 -o sref_precip.x sref_precip.f /apps/ops/prod/libs/intel/19.1.3.304/ip/4.0.0/lib64/libip_4.a /lfs/h2/emc/eib/save/hang.lei/forgdit/nco_wcoss2/install2/intel-19.1.3.304/w3emc/2.12.0/lib64/libw3emc_4.a /apps/ops/prod/libs/intel/19.1.3.304/bacio/2.4.1/lib/libbacio_4.a  /apps/ops/prod/libs/intel/19.1.3.304/sp/2.4.0/lib64/libsp_4.a /lfs/h2/emc/eib/save/hang.lei/forgdit/nco_wcoss2/install2/intel-19.1.3.304/g2/3.5.1/lib64/libg2_4.a /apps/spack/jasper/2.0.25/intel/19.1.3.304/sjib74krrorkyczqpqah4tvewmlnqdx4/lib64/libjasper.a /apps/spack/libpng/1.6.37/intel/19.1.3.304/4ohkronuhlyherusoszzrmur5ewvlwzh/lib/libpng.a /apps/spack/libjpeg/9c/intel/19.1.3.304/jkr3isi257ktoouprwaxcn4twtye747z/lib/libjpeg.a /apps/spack/zlib/1.2.11/intel/19.1.3.304/hjotqkckeoyt6j6tibalwzrlfljcjtdh/lib/libz.a

But the same errors occur.

I reverted back to the prod install of g2/3.4.5 (and ip/4.0.0 and para installation of w3emc/2.12.0) and EVS compiled correctly.

Hang-Lei-NOAA commented 2 months ago

Hi, Shelley

Thanks, Please use the w3emc/2.12.0 and g2/3.4.5 on wcoss2 for now.

This g2/3.5.1 was for UPP. AQM also reported error on this. We will further investigate it.

Thanks, Hang

On Mon, Jul 22, 2024 at 2:23 PM Shelley Melchior @.***> wrote:

Thanks Hang! I tried to recompile using your installation. Here is the make command:

ftn -g -traceback -O3 -auto -convert big_endian -I /lfs/h2/emc/eib/save/hang.lei/forgdit/nco_wcoss2/install2/intel-19.1.3.304/g2/3.5.1/include_4 -o sref_precip.x sref_precip.f /apps/ops/prod/libs/intel/19.1.3.304/ip/4.0.0/lib64/libip_4.a /lfs/h2/emc/eib/save/hang.lei/forgdit/nco_wcoss2/install2/intel-19.1.3.304/w3emc/2.12.0/lib64/libw3emc_4.a /apps/ops/prod/libs/intel/19.1.3.304/bacio/2.4.1/lib/libbacio_4.a /apps/ops/prod/libs/intel/19.1.3.304/sp/2.4.0/lib64/libsp_4.a /lfs/h2/emc/eib/save/hang.lei/forgdit/nco_wcoss2/install2/intel-19.1.3.304/g2/3.5.1/lib64/libg2_4.a /apps/spack/jasper/2.0.25/intel/19.1.3.304/sjib74krrorkyczqpqah4tvewmlnqdx4/lib64/libjasper.a /apps/spack/libpng/1.6.37/intel/19.1.3.304/4ohkronuhlyherusoszzrmur5ewvlwzh/lib/libpng.a /apps/spack/libjpeg/9c/intel/19.1.3.304/jkr3isi257ktoouprwaxcn4twtye747z/lib/libjpeg.a /apps/spack/zlib/1.2.11/intel/19.1.3.304/hjotqkckeoyt6j6tibalwzrlfljcjtdh/lib/libz.a

But the same errors occur.

I reverted back to the prod install of g2/3.4.5 (and ip/4.0.0 and para installation of w3emc/2.12.0) and EVS compiled correctly.

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/NCEPLIBS-w3emc/issues/234#issuecomment-2243556420, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFDADNPRAMGJLABFOQDZNVE2LAVCNFSM6AAAAABJ4KMXMKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBTGU2TMNBSGA . You are receiving this because you were mentioned.Message ID: @.***>

edwardhartnett commented 2 months ago

Could it be that two different versions of jpeg are involved? That is, g2 was build with one version, and some other library includes a different version?

Hang-Lei-NOAA commented 2 months ago

Not possible. they all use system installed libraries. There is a default rule for wcoss installations that using system library is a priority. Only one jasper, jpeg, png, bacio, library available in the environment.

On Mon, Jul 22, 2024 at 2:45 PM Edward Hartnett @.***> wrote:

Could it be that two different versions of jpeg are involved? That is, g2 was build with one version, and some other library includes a different version?

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/NCEPLIBS-w3emc/issues/234#issuecomment-2243593502, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFCGSSBOYL2MK5LQ2PLZNVHK3AVCNFSM6AAAAABJ4KMXMKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBTGU4TGNJQGI . You are receiving this because you were mentioned.Message ID: @.***>

ShelleyMelchior-NOAA commented 2 months ago

Thanks for the help and discussion! EVSv2 will plan for ip/4.0.0 and w3emc/2.12.0, while using g2/3.4.5.

@Hang-Lei-NOAA , can you let me know whenw3emc/2.12.0 has been promoted to a prod installation?

Hang-Lei-NOAA commented 2 months ago

Not sure. But will let you know when NCO plans to do so.

On Mon, Jul 22, 2024 at 3:49 PM Shelley Melchior @.***> wrote:

Thanks for the help and discussion! EVSv2 will plan for ip/4.0.0 and w3emc/2.12.0, while using g2/3.4.5.

@Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA , can you let me know whenw3emc/2.12.0 has been promoted to a prod installation?

— Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/NCEPLIBS-w3emc/issues/234#issuecomment-2243696565, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFENHVOYN74QZ7X6ICDZNVO3VAVCNFSM6AAAAABJ4KMXMKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENBTGY4TMNJWGU . You are receiving this because you were mentioned.Message ID: @.***>