search

NOAA-EMC / hpc-stack

Create a software stack for HPC's
GNU Lesser General Public License v2.1
30 stars 36 forks source link

eccodes #247

Closed edwardhartnett closed 1 year ago

edwardhartnett commented 3 years ago

At @arunchawla-NOAA 's request @Hang-Lei-NOAA can you install eccodes on hera so that we can get some people to play with it.

Please notify @AliciaBentley-NOAA and @LoganDawson-NOAA when it's installed on hera.

Hang-Lei-NOAA commented 3 years ago

Yes, I will install it today

On Tue, May 25, 2021 at 4:24 PM Edward Hartnett @.***> wrote:

Assigned #247 https://github.com/NOAA-EMC/hpc-stack/issues/247 to @Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/hpc-stack/issues/247#event-4795048028, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFGKHVNIV6B2S6OACEDTPQBRLANCNFSM45QEP4RA .

Hang-Lei-NOAA commented 3 years ago

The eccodes/2.22.0 has been added to hpc-stack on Hera, with intel compiler. Please try and give feedback if you have any concerns.

/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/modulefiles/mpi/intel/18.0.5.274/impi/2018.0.4

atlas/ecmwf-0.24.1 (D) esmf/8_1_0_beta_snapshot_27-debug esmf/8_1_0_beta_snapshot_47 fms/noaa-gfdl-2020.04.03 ncio/1.0.0 (D) pio/2.5.2 (D) upp/10.0.4 wgrib2/2.0.8 (D) atlas/jcsda-release-stable esmf/8_1_0_beta_snapshot_27 esmf/8_1_1-debug (D) fms/2020.04.03 (D) nemsio/2.5.2 (D) upp/10.0.0 upp/10.0.5 wrf_io/1.1.1

-bash-4.2$ module show eccodes/2.22.0

/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/modulefiles/mpi/intel/18.0.5.274/impi/2018.0.4/eccodes/2.22.0.lua:

help([[]]) conflict("eccodes") prepend_path("PATH","/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/eccodes/2.22.0/bin") prepend_path("LD_LIBRARY_PATH","/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/eccodes/2.22.0/lib64") prepend_path("DYLD_LIBRARY_PATH","/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/eccodes/2.22.0/lib64") prepend_path("CPATH","/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/eccodes/2.22.0/include") setenv("ECCODES","/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/eccodes/2.22.0") setenv("ECCODES_ROOT","/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/eccodes/2.22.0") setenv("ECCODES_INCLUDES","/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/eccodes/2.22.0/include") setenv("ECCODES_LIBRARIES","/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/eccodes/2.22.0/lib64") setenv("ECCODES_VERSION","2.22.0") setenv("eccodes","/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/eccodes/2.22.0") setenv("eccodes_ROOT","/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/eccodes/2.22.0") setenv("eccodes_INCLUDES","/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/eccodes/2.22.0/include") setenv("eccodes_LIBRARIES","/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/eccodes/2.22.0/lib64") setenv("eccodes_VERSION","2.22.0") whatis("Name: eccodes") whatis("Version: 2.22.0") whatis("Category: library") whatis("Description: eccodes library")

On Tue, May 25, 2021 at 4:29 PM Hang Lei - NOAA Affiliate @.***> wrote:

Yes, I will install it today

On Tue, May 25, 2021 at 4:24 PM Edward Hartnett @.***> wrote:

Assigned #247 https://github.com/NOAA-EMC/hpc-stack/issues/247 to @Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/hpc-stack/issues/247#event-4795048028, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFGKHVNIV6B2S6OACEDTPQBRLANCNFSM45QEP4RA .

LoganDawson-NOAA commented 3 years ago

@Hang-Lei-NOAA thanks for installing eccodes

I took a look, and I don't see anything related to the Python API available from eccodes in the library path. I would have expected to see a directory such as the following path if the python API was enabled: /scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/eccodes/2.22.0/lib64/python3.6/site-packages

I attempted to import eccodes within python after running module load eccodes/2.22.0, but eccodes was not found:

>>> import eccodes
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'eccodes'
>>>

I confirmed that the module load eccodes/2.22.0 command doesn't prepend any directory to the PYTHONPATH env variable to make eccodes classes available for python scripting.

Hang-Lei-NOAA commented 3 years ago

Hi, Logan

The python api, pip3, was forbidden to download on the noaa machine. Therefore, it is just the eccodes installation.

Thanks, Hang

On Thu, Jun 3, 2021 at 11:42 AM Logan Dawson @.***> wrote:

@Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA thanks for installing eccodes

I took a look, and I don't see anything related to the Python API available from eccodes in the library path. I would have expected to see a directory such as the following path if the python API was enabled:

/scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/intel-18.0.5.274/impi-2018.0.4/eccodes/2.22.0/lib64/ python3.6/site-packages

I attempted to import eccodes within python after running module load eccodes/2.22.0, but eccodes was not found:

import eccodes Traceback (most recent call last): File "", line 1, in ModuleNotFoundError: No module named 'eccodes'

I confirmed that the module load eccodes/2.22.0 command doesn't prepend any directory to the PYTHONPATH env variable to make eccodes classes available for python scripting.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/hpc-stack/issues/247#issuecomment-853968114, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFCO643UCDEC2GUL6YTTQ6PG5ANCNFSM45QEP4RA .

LoganDawson-NOAA commented 3 years ago

Thanks for that clarification, @Hang-Lei-NOAA

On a larger note, and I'll need to touch base with @arunchawla-NOAA offline about this... is the idea that we'll be using the eccodes Python API to read GRIB data on WCOSS2? If so, that's a new and unexpected approach to me.

I thought we were still planning to use pygrib, which has ECCODES library as a dependency. For this approach on the current WCOSS, I've never actually directly tested eccodes itself. The eccodes dependency is essentially invisible because we only import pygrib. I'll follow up with Arun on Slack.

Hang-Lei-NOAA commented 3 years ago

Hi, Logan BTW, the hpc-stack libraries are used by : module use /scratch2/NCEPDEV/nwprod/hpc-stack/libs/hpc-stack/modulefiles/stack module load hpc module load hpc-intel module load hpc-impi

Then you can load what library you want to use. Thanks

On Thu, Jun 3, 2021 at 11:50 AM Logan Dawson @.***> wrote:

Thanks for that clarification, @Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA

On a larger note, and I'll need to touch base with @arunchawla-NOAA https://github.com/arunchawla-NOAA offline about this... is the idea that we'll be using the eccodes Python API to read GRIB data on WCOSS2? If so, that's a new and unexpected approach to me.

I thought we were still planning to use pygrib, which has ECCODES library as a dependency. For this approach on the current WCOSS, I've never actually directly tested eccodes itself. The eccodes dependency is essentially invisible because we only import pygrib. I'll follow up with Arun on Slack.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/hpc-stack/issues/247#issuecomment-853975203, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFH27UVUNDEKCTF66HTTQ6QFTANCNFSM45QEP4RA .

LoganDawson-NOAA commented 3 years ago

Thanks for the info @Hang-Lei-NOAA.

With some additional context from @arunchawla-NOAA, I wonder if we can try this installation and test from Orion? Since pip3 is required to install the python bindings and that external connection is blocked from Hera, we can't really test this eccodes python interface on Hera.

I have an Orion account and can test reading GRIB data if hpc-stack/eccodes can be installed and if pip3 isn't blocked there.

aerorahul commented 3 years ago

I don't see an eccodes build script added to hpc-stack. How was eccodes installed on Hera without it being built via hpc-stack? Did we just violate the rule that we will not do ad-hoc installations without version control.

Hang-Lei-NOAA commented 3 years ago

I have a testing version that added the eccode. After further test, I will push it to the repo. This was an urgent request from Arun and developer to add this for testing purposes with existing hpc-stack modules. The testing has finished. I will remove this eccode module right now, and install it after pull to the repo.

On Thu, Jul 8, 2021 at 11:28 AM Rahul Mahajan @.***> wrote:

I don't see an eccodes build script added to hpc-stack. How was eccodes installed on Hera without it being built via hpc-stack? Did we just violate the rule that we will not do ad-hoc installations without version control.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/hpc-stack/issues/247#issuecomment-876534358, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFDMUEO2EBUOP2XVJCLTWW7YDANCNFSM45QEP4RA .

arunchawla-NOAA commented 3 years ago

All installation and development should be marked in repositories. We really cannot do anything outside repositories. Please factor in time for doing this right. The work can be forks as a bare minimum, but not being in repos is as good as not being done because there is no way for us to keep track

On Thu, Jul 8, 2021 at 11:36 AM Hang-Lei-NOAA @.***> wrote:

I have a testing version that added the eccode. After further test, I will push it to the repo. This was an urgent request from Arun and developer to add this for testing purposes with existing hpc-stack modules. The testing has finished. I will remove this eccode module right now, and install it after pull to the repo.

On Thu, Jul 8, 2021 at 11:28 AM Rahul Mahajan @.***> wrote:

I don't see an eccodes build script added to hpc-stack. How was eccodes installed on Hera without it being built via hpc-stack? Did we just violate the rule that we will not do ad-hoc installations without version control.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/NOAA-EMC/hpc-stack/issues/247#issuecomment-876534358 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AKWSMFDMUEO2EBUOP2XVJCLTWW7YDANCNFSM45QEP4RA

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/hpc-stack/issues/247#issuecomment-876540658, or unsubscribe https://github.com/notifications/unsubscribe-auth/AL5NYI5J66OMCSU7W32KHEDTWXAXRANCNFSM45QEP4RA .

--

Arun Chawla Chief Engineering & Implementation Branch Room 2083 National Center for Weather & Climate Prediction 5830 University Research Court College Park, MD 20740 Phone: 301-683-3740 Mobile: 240-564-5675 Fax: 301-683-3703

Hang-Lei-NOAA commented 3 years ago

Yes, I will correct this as soon as possible.

On Thu, Jul 8, 2021 at 11:40 AM arun chawla @.***> wrote:

All installation and development should be marked in repositories. We really cannot do anything outside repositories. Please factor in time for doing this right. The work can be forks as a bare minimum, but not being in repos is as good as not being done because there is no way for us to keep track

On Thu, Jul 8, 2021 at 11:36 AM Hang-Lei-NOAA @.***> wrote:

I have a testing version that added the eccode. After further test, I will push it to the repo. This was an urgent request from Arun and developer to add this for testing purposes with existing hpc-stack modules. The testing has finished. I will remove this eccode module right now, and install it after pull to the repo.

On Thu, Jul 8, 2021 at 11:28 AM Rahul Mahajan @.***> wrote:

I don't see an eccodes build script added to hpc-stack. How was eccodes installed on Hera without it being built via hpc-stack? Did we just violate the rule that we will not do ad-hoc installations without version control.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/NOAA-EMC/hpc-stack/issues/247#issuecomment-876534358 , or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AKWSMFDMUEO2EBUOP2XVJCLTWW7YDANCNFSM45QEP4RA

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/NOAA-EMC/hpc-stack/issues/247#issuecomment-876540658 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AL5NYI5J66OMCSU7W32KHEDTWXAXRANCNFSM45QEP4RA

.

--

Arun Chawla Chief Engineering & Implementation Branch Room 2083 National Center for Weather & Climate Prediction 5830 University Research Court College Park, MD 20740 Phone: 301-683-3740 Mobile: 240-564-5675 Fax: 301-683-3703

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/hpc-stack/issues/247#issuecomment-876543495, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFC7KFXFJ5KO45PMD3TTWXBF3ANCNFSM45QEP4RA .

edwardhartnett commented 3 years ago

@Hang-Lei-NOAA what's the status of this issue? Will it be ready for the upcoming hpc-stack 1.2.0 release?

Hang-Lei-NOAA commented 3 years ago

No, it's not. I hasn’t prepare this pr.

On Tue, Jul 27, 2021 at 9:34 AM Edward Hartnett @.***> wrote:

@Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA what's the status of this issue? Will it be ready for the upcoming hpc-stack 1.2.0 release?

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/hpc-stack/issues/247#issuecomment-887517658, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFH6W3X3SI35OKALJDTTZ2YWLANCNFSM45QEP4RA .

edwardhartnett commented 3 years ago

OK, we will bump this to the 1.3.0 release.

aerorahul commented 1 year ago

Is there a plan to still work on this?

Hang-Lei-NOAA commented 1 year ago

No. eccodes have been clearly declined by NCO for use in models.

aerorahul commented 1 year ago

Ok. Then close this issue?

Hang-Lei-NOAA commented 1 year ago

Yes

climbfuji commented 1 year ago

@Hang-Lei-NOAA I didn't know that eccodes was forbidden on wcoss2. Are you making sure that the spack-stack environment (or whatever gets installed there) doesn't have eccodes on wcoss2?

arunchawla-NOAA commented 1 year ago

eccodes is not approved for operations. Why are we using eccodes ?

climbfuji commented 1 year ago

eccodes is not approved for operations. Why are we using eccodes ?

You don't use eccodes. We have a unified spack-stack environment that contains everything possibly needed for JEDI, UFS, soon GEOS, NEPTUNE. That gets installed in containers, on R&D HPCs, the cloud, but not on WCOSS2. The unified environment is a large superset of what is needed in operations (and in R&D for UFS applications), including eccodes for certain JEDI applications and converters (but not for the UFS).

arunchawla-NOAA commented 1 year ago

ok thanks for that Dom