search

NOAA-EMC / global-workflow

Global Superstructure/Workflow supporting the Global Forecast System (GFS)
https://global-workflow.readthedocs.io/en/latest
GNU Lesser General Public License v3.0
75 stars 168 forks source link

Port to Orion #5

Closed KateFriedman-NOAA closed 4 years ago

KateFriedman-NOAA commented 4 years ago

Port global-workflow and components to Mississippi machine when available (called Orion).

Branch for work: port2orion (made off of feature/gfsv16b)

Original issue on VLab: https://vlab.ncep.noaa.gov/redmine/issues/67194

Established "glopara" space: /work/noaa/global/glopara

To do list:

KateFriedman-NOAA commented 4 years ago

Sync merged feature/gfsv16b into port2orion after testing connection to GitHub on Orion.

KateFriedman-NOAA commented 4 years ago

Added new ORION files to workflow: module files (using temp NCEPLIBS) and env file Added ORION to setup scripts, machine-setup.sh, link script, and configs...used temporary paths for group account space, will need to update later.

Tested setup scripts and they successfully created an EXPDIR, COMROT, configs in EXPDIR, and a rocoto xml in EXPDIR. Don't yet have rocoto available to test xml further.

KateFriedman-NOAA commented 4 years ago

Was able to get gaussian_sfcanl.exe to build by manually running each step of build_gaussian_sfcanl.sh but when I run build_gaussian_sfcanl.sh it stops while sourcing the modulefile. Need to investigate further.

KateFriedman-NOAA commented 4 years ago

Found cause of build issue for gaussian_sfcanl.exe...variable that defines path to HDF5 libraries in module file had to be updated. The build_gaussian_sfcanl.sh script now works on Orion.

KateFriedman-NOAA commented 4 years ago

Need w3emc_para/2.4.0 library on Orion to build gfs_bufr. @Hang-Lei-NOAA When will NCEPLIBS be set up on Orion? Have you and others from the NCEPLIBS team gotten Orion accounts?

@GeorgeVandenberghe-NOAA Would you be able to add the w3emc_para/2.4.0 library to your set on Orion until the official NCEPLIBS is available? The changes for v16 also require these new libraries if you can add them too:

Thanks!

GeorgeVandenberghe-NOAA commented 4 years ago

I am having trouble getting parallel netcdf to RUN on Orion. Have not run it down yet. It works when the parallel API is disabled (through namelist) at runtime.

Have asked others about their results but no response yet.

I need a source for w3emc_para.

On Thu, Mar 19, 2020 at 10:49 AM Kate Friedman notifications@github.com wrote:

Need w3emc_para/2.4.0 library on Orion to build gfs_bufr. @Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA When will NCEPLIBS be set up on Orion? Have you and others from the NCEPLIBS team get Orion accounts?

@GeorgeVandenberghe-NOAA https://github.com/GeorgeVandenberghe-NOAA Would you be able to add the w3emc_para/2.4.0 library to your set on Orion until the official NCEPLIBS is available? The changes for v16 also require these new libraries if you can add them too:

  • hdf5_parallel/1.10.6
  • netcdf_parallel/4.7.4
  • w3emc_para/2.4.0

Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/global-workflow/issues/5#issuecomment-601221465, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANDS4FRBLNRVNHLHMYWU5ATRIIWFRANCNFSM4J4BLX4Q .

--

George W Vandenberghe

IMSG at NOAA/NWS/NCEP/EMC

5830 University Research Ct., Rm. 2141

College Park, MD 20740

George.Vandenberghe@noaa.gov

301-683-3769(work) 3017751547(cell)

Hang-Lei-NOAA commented 4 years ago

Hi, All,

I will coordinate with MSU to address my access to Orion. Once it is done, it will be installed the same day.

Thanks, Hang

On Thu, Mar 19, 2020 at 11:04 AM GeorgeVandenberghe-NOAA < notifications@github.com> wrote:

I am having trouble getting parallel netcdf to RUN on Orion. Have not run it down yet. It works when the parallel API is disabled (through namelist) at runtime.

Have asked others about their results but no response yet.

I need a source for w3emc_para.

On Thu, Mar 19, 2020 at 10:49 AM Kate Friedman notifications@github.com wrote:

Need w3emc_para/2.4.0 library on Orion to build gfs_bufr. @Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA When will NCEPLIBS be set up on Orion? Have you and others from the NCEPLIBS team get Orion accounts?

@GeorgeVandenberghe-NOAA https://github.com/GeorgeVandenberghe-NOAA Would you be able to add the w3emc_para/2.4.0 library to your set on Orion until the official NCEPLIBS is available? The changes for v16 also require these new libraries if you can add them too:

  • hdf5_parallel/1.10.6
  • netcdf_parallel/4.7.4
  • w3emc_para/2.4.0

Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/NOAA-EMC/global-workflow/issues/5#issuecomment-601221465 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ANDS4FRBLNRVNHLHMYWU5ATRIIWFRANCNFSM4J4BLX4Q

.

--

George W Vandenberghe

IMSG at NOAA/NWS/NCEP/EMC

5830 University Research Ct., Rm. 2141

College Park, MD 20740

George.Vandenberghe@noaa.gov

301-683-3769(work) 3017751547(cell)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/global-workflow/issues/5#issuecomment-601230268, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFDGHLJMPDZQ5GXH3O3RIIX77ANCNFSM4J4BLX4Q .

KateFriedman-NOAA commented 4 years ago

Thanks @Hang-Lei-NOAA and @GeorgeVandenberghe-NOAA ! I just had a tag-up with Arun and will be pausing on Orion for a bit until some issues are resolved (rstprod available, official NCEPLIBS available, group space for me fixed). @Hang-Lei-NOAA Let me know when you gain access to Orion and have NCEPLIBS installed, no urgent rush on this, thanks!

GeorgeVandenberghe-NOAA commented 4 years ago

My NCEPLIBS should work. I know of at least two issues where it IS broken, parallel netcdf API, and w3emc-para unavailable. The second should be straightforward. The first will take a long time to run down unless I get help and a testcase smaller than a global GCM which it's probably time to uuhh.. reach out and do.

On Thu, Mar 19, 2020 at 11:44 AM Kate Friedman notifications@github.com wrote:

Thanks @Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA and @GeorgeVandenberghe-NOAA https://github.com/GeorgeVandenberghe-NOAA ! I just had a tag-up with Arun and will be pausing on Orion for a bit until some issues are resolved (rstprod available, official NCEPLIBS available, group space for me fixed). @Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA Let me know when you gain access to Orion and have NCEPLIBS installed, no urgent rush on this, thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/global-workflow/issues/5#issuecomment-601252382, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANDS4FS6YBHJOZE6UOTYWXTRII4URANCNFSM4J4BLX4Q .

--

George W Vandenberghe

IMSG at NOAA/NWS/NCEP/EMC

5830 University Research Ct., Rm. 2141

College Park, MD 20740

George.Vandenberghe@noaa.gov

301-683-3769(work) 3017751547(cell)

GeorgeVandenberghe-NOAA commented 4 years ago

My parallel netcdf issues have been resolved. The problem was I built with impi/2018 and ran with impi.2019. switching back to impi/2018 which is less stable at huge task counts, allowed it to run and write compressed netcdf in parallel.

Bottom line, I got gfsv162 to build and write hetcdf fast in parallel

On Thu, Mar 19, 2020 at 11:02 AM George Vandenberghe - NOAA Affiliate < george.vandenberghe@noaa.gov> wrote:

I am having trouble getting parallel netcdf to RUN on Orion. Have not run it down yet. It works when the parallel API is disabled (through namelist) at runtime.

Have asked others about their results but no response yet.

I need a source for w3emc_para.

On Thu, Mar 19, 2020 at 10:49 AM Kate Friedman notifications@github.com wrote:

Need w3emc_para/2.4.0 library on Orion to build gfs_bufr. @Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA When will NCEPLIBS be set up on Orion? Have you and others from the NCEPLIBS team get Orion accounts?

@GeorgeVandenberghe-NOAA https://github.com/GeorgeVandenberghe-NOAA Would you be able to add the w3emc_para/2.4.0 library to your set on Orion until the official NCEPLIBS is available? The changes for v16 also require these new libraries if you can add them too:

  • hdf5_parallel/1.10.6
  • netcdf_parallel/4.7.4
  • w3emc_para/2.4.0

Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/global-workflow/issues/5#issuecomment-601221465, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANDS4FRBLNRVNHLHMYWU5ATRIIWFRANCNFSM4J4BLX4Q .

--

George W Vandenberghe

IMSG at NOAA/NWS/NCEP/EMC

5830 University Research Ct., Rm. 2141

College Park, MD 20740

George.Vandenberghe@noaa.gov

301-683-3769(work) 3017751547(cell)

--

George W Vandenberghe

IMSG at NOAA/NWS/NCEP/EMC

5830 University Research Ct., Rm. 2141

College Park, MD 20740

George.Vandenberghe@noaa.gov

301-683-3769(work) 3017751547(cell)

GeorgeVandenberghe-NOAA commented 4 years ago

Hang. Do you have a source or git location (preferred) for w3emc_para/2.4 so I can build it in my portable location everywhere including orion.

On Thu, Mar 19, 2020 at 11:02 AM George Vandenberghe - NOAA Affiliate < george.vandenberghe@noaa.gov> wrote:

I am having trouble getting parallel netcdf to RUN on Orion. Have not run it down yet. It works when the parallel API is disabled (through namelist) at runtime.

Have asked others about their results but no response yet.

I need a source for w3emc_para.

On Thu, Mar 19, 2020 at 10:49 AM Kate Friedman notifications@github.com wrote:

Need w3emc_para/2.4.0 library on Orion to build gfs_bufr. @Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA When will NCEPLIBS be set up on Orion? Have you and others from the NCEPLIBS team get Orion accounts?

@GeorgeVandenberghe-NOAA https://github.com/GeorgeVandenberghe-NOAA Would you be able to add the w3emc_para/2.4.0 library to your set on Orion until the official NCEPLIBS is available? The changes for v16 also require these new libraries if you can add them too:

  • hdf5_parallel/1.10.6
  • netcdf_parallel/4.7.4
  • w3emc_para/2.4.0

Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/global-workflow/issues/5#issuecomment-601221465, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANDS4FRBLNRVNHLHMYWU5ATRIIWFRANCNFSM4J4BLX4Q .

--

George W Vandenberghe

IMSG at NOAA/NWS/NCEP/EMC

5830 University Research Ct., Rm. 2141

College Park, MD 20740

George.Vandenberghe@noaa.gov

301-683-3769(work) 3017751547(cell)

--

George W Vandenberghe

IMSG at NOAA/NWS/NCEP/EMC

5830 University Research Ct., Rm. 2141

College Park, MD 20740

George.Vandenberghe@noaa.gov

301-683-3769(work) 3017751547(cell)

Hang-Lei-NOAA commented 4 years ago

hi George

The w3emc_para/2.4 has no change in w3emc library. It just compiled with the para-netcdf.

You can get the tar file from GitHub necplibs-w3emc repo. With tag2.4.0

Thanks Hang

On Thu, Mar 19, 2020 at 8:45 PM GeorgeVandenberghe-NOAA < notifications@github.com> wrote:

Hang. Do you have a source or git location (preferred) for w3emc_para/2.4 so I can build it in my portable location everywhere including orion.

On Thu, Mar 19, 2020 at 11:02 AM George Vandenberghe - NOAA Affiliate < george.vandenberghe@noaa.gov> wrote:

I am having trouble getting parallel netcdf to RUN on Orion. Have not run it down yet. It works when the parallel API is disabled (through namelist) at runtime.

Have asked others about their results but no response yet.

I need a source for w3emc_para.

On Thu, Mar 19, 2020 at 10:49 AM Kate Friedman <notifications@github.com

wrote:

Need w3emc_para/2.4.0 library on Orion to build gfs_bufr. @Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA When will NCEPLIBS be set up on Orion? Have you and others from the NCEPLIBS team get Orion accounts?

@GeorgeVandenberghe-NOAA https://github.com/GeorgeVandenberghe-NOAA Would you be able to add the w3emc_para/2.4.0 library to your set on Orion until the official NCEPLIBS is available? The changes for v16 also require these new libraries if you can add them too:

  • hdf5_parallel/1.10.6
  • netcdf_parallel/4.7.4
  • w3emc_para/2.4.0

Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/NOAA-EMC/global-workflow/issues/5#issuecomment-601221465 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ANDS4FRBLNRVNHLHMYWU5ATRIIWFRANCNFSM4J4BLX4Q

.

--

George W Vandenberghe

IMSG at NOAA/NWS/NCEP/EMC

5830 University Research Ct., Rm. 2141 https://www.google.com/maps/search/5830+University+Research+Ct.,+Rm.+2141+%0D%0D%0A+%0D%0D%0A+College+Park,+MD+20740?entry=gmail&source=g

https://www.google.com/maps/search/5830+University+Research+Ct.,+Rm.+2141+%0D%0D%0A+%0D%0D%0A+College+Park,+MD+20740?entry=gmail&source=g

https://www.google.com/maps/search/5830+University+Research+Ct.,+Rm.+2141+%0D%0D%0A+%0D%0D%0A+College+Park,+MD+20740?entry=gmail&source=g> College Park, MD 20740 https://www.google.com/maps/search/5830+University+Research+Ct.,+Rm.+2141+%0D%0D%0A+%0D%0D%0A+College+Park,+MD+20740?entry=gmail&source=g

George.Vandenberghe@noaa.gov

301-683-3769(work) 3017751547(cell)

--

George W Vandenberghe

IMSG at NOAA/NWS/NCEP/EMC

5830 University Research Ct., Rm. 2141 https://www.google.com/maps/search/5830+University+Research+Ct.,+Rm.+2141+%0D%0D%0A+%0D%0D%0ACollege+Park,+MD+20740?entry=gmail&source=g

https://www.google.com/maps/search/5830+University+Research+Ct.,+Rm.+2141+%0D%0D%0A+%0D%0D%0ACollege+Park,+MD+20740?entry=gmail&source=g College Park, MD 20740 https://www.google.com/maps/search/5830+University+Research+Ct.,+Rm.+2141+%0D%0D%0A+%0D%0D%0ACollege+Park,+MD+20740?entry=gmail&source=g

George.Vandenberghe@noaa.gov

301-683-3769(work) 3017751547(cell)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/global-workflow/issues/5#issuecomment-601480552, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFEHWGEXNUFEKU7LCULRIK4DNANCNFSM4J4BLX4Q .

GeorgeVandenberghe-NOAA commented 4 years ago

What is the exact syntax for this git from github?

On Thu, Mar 19, 2020 at 9:04 PM Hang-Lei-NOAA notifications@github.com wrote:

hi George

The w3emc_para/2.4 has no change in w3emc library. It just compiled with the para-netcdf.

You can get the tar file from GitHub necplibs-w3emc repo. With tag2.4.0

Thanks Hang

On Thu, Mar 19, 2020 at 8:45 PM GeorgeVandenberghe-NOAA < notifications@github.com> wrote:

Hang. Do you have a source or git location (preferred) for w3emc_para/2.4 so I can build it in my portable location everywhere including orion.

On Thu, Mar 19, 2020 at 11:02 AM George Vandenberghe - NOAA Affiliate < george.vandenberghe@noaa.gov> wrote:

I am having trouble getting parallel netcdf to RUN on Orion. Have not run it down yet. It works when the parallel API is disabled (through namelist) at runtime.

Have asked others about their results but no response yet.

I need a source for w3emc_para.

On Thu, Mar 19, 2020 at 10:49 AM Kate Friedman < notifications@github.com

wrote:

Need w3emc_para/2.4.0 library on Orion to build gfs_bufr. @Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA When will NCEPLIBS be set up on Orion? Have you and others from the NCEPLIBS team get Orion accounts?

@GeorgeVandenberghe-NOAA https://github.com/GeorgeVandenberghe-NOAA Would you be able to add the w3emc_para/2.4.0 library to your set on Orion until the official NCEPLIBS is available? The changes for v16 also require these new libraries if you can add them too:

  • hdf5_parallel/1.10.6
  • netcdf_parallel/4.7.4
  • w3emc_para/2.4.0

Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <

https://github.com/NOAA-EMC/global-workflow/issues/5#issuecomment-601221465

,

or unsubscribe <

https://github.com/notifications/unsubscribe-auth/ANDS4FRBLNRVNHLHMYWU5ATRIIWFRANCNFSM4J4BLX4Q

.

--

George W Vandenberghe

IMSG at NOAA/NWS/NCEP/EMC

5830 University Research Ct., Rm. 2141 https://www.google.com/maps/search/5830+University+Research+Ct .,+Rm.+2141+%0D%0D%0A+%0D%0D%0A+College+Park,+MD+20740?entry=gmail&source=g

https://www.google.com/maps/search/5830+University+Research+Ct .,+Rm.+2141+%0D%0D%0A+%0D%0D%0A+College+Park,+MD+20740?entry=gmail&source=g

https://www.google.com/maps/search/5830+University+Research+Ct .,+Rm.+2141+%0D%0D%0A+%0D%0D%0A+College+Park,+MD+20740?entry=gmail&source=g> College Park, MD 20740 https://www.google.com/maps/search/5830+University+Research+Ct .,+Rm.+2141+%0D%0D%0A+%0D%0D%0A+College+Park,+MD+20740?entry=gmail&source=g

George.Vandenberghe@noaa.gov

301-683-3769(work) 3017751547(cell)

--

George W Vandenberghe

IMSG at NOAA/NWS/NCEP/EMC

5830 University Research Ct., Rm. 2141 https://www.google.com/maps/search/5830+University+Research+Ct .,+Rm.+2141+%0D%0D%0A+%0D%0D%0ACollege+Park,+MD+20740?entry=gmail&source=g

https://www.google.com/maps/search/5830+University+Research+Ct .,+Rm.+2141+%0D%0D%0A+%0D%0D%0ACollege+Park,+MD+20740?entry=gmail&source=g College Park, MD 20740 https://www.google.com/maps/search/5830+University+Research+Ct .,+Rm.+2141+%0D%0D%0A+%0D%0D%0ACollege+Park,+MD+20740?entry=gmail&source=g

George.Vandenberghe@noaa.gov

301-683-3769(work) 3017751547(cell)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/NOAA-EMC/global-workflow/issues/5#issuecomment-601480552 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AKWSMFEHWGEXNUFEKU7LCULRIK4DNANCNFSM4J4BLX4Q

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/global-workflow/issues/5#issuecomment-601484194, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANDS4FROWFG25MOQSENPW2TRIK6K3ANCNFSM4J4BLX4Q .

--

George W Vandenberghe

IMSG at NOAA/NWS/NCEP/EMC

5830 University Research Ct., Rm. 2141

College Park, MD 20740

George.Vandenberghe@noaa.gov

301-683-3769(work) 3017751547(cell)

Hang-Lei-NOAA commented 4 years ago

Hi, George

git clone https://github.com/NOAA-EMC/NCEPLIBS-w3emc.git git tag -l git checkout w3emc_v2.4.0

you will get that tag.

Thanks, Hang

On Fri, Mar 20, 2020 at 10:33 AM GeorgeVandenberghe-NOAA < notifications@github.com> wrote:

What is the exact syntax for this git from github?

On Thu, Mar 19, 2020 at 9:04 PM Hang-Lei-NOAA notifications@github.com wrote:

hi George

The w3emc_para/2.4 has no change in w3emc library. It just compiled with the para-netcdf.

You can get the tar file from GitHub necplibs-w3emc repo. With tag2.4.0

Thanks Hang

On Thu, Mar 19, 2020 at 8:45 PM GeorgeVandenberghe-NOAA < notifications@github.com> wrote:

Hang. Do you have a source or git location (preferred) for w3emc_para/2.4 so I can build it in my portable location everywhere including orion.

On Thu, Mar 19, 2020 at 11:02 AM George Vandenberghe - NOAA Affiliate < george.vandenberghe@noaa.gov> wrote:

I am having trouble getting parallel netcdf to RUN on Orion. Have not run it down yet. It works when the parallel API is disabled (through namelist) at runtime.

Have asked others about their results but no response yet.

I need a source for w3emc_para.

On Thu, Mar 19, 2020 at 10:49 AM Kate Friedman < notifications@github.com

wrote:

Need w3emc_para/2.4.0 library on Orion to build gfs_bufr. @Hang-Lei-NOAA https://github.com/Hang-Lei-NOAA When will NCEPLIBS be set up on Orion? Have you and others from the NCEPLIBS team get Orion accounts?

@GeorgeVandenberghe-NOAA < https://github.com/GeorgeVandenberghe-NOAA> Would you be able to add the w3emc_para/2.4.0 library to your set on Orion until the official NCEPLIBS is available? The changes for v16 also require these new libraries if you can add them too:

  • hdf5_parallel/1.10.6
  • netcdf_parallel/4.7.4
  • w3emc_para/2.4.0

Thanks!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <

https://github.com/NOAA-EMC/global-workflow/issues/5#issuecomment-601221465

,

or unsubscribe <

https://github.com/notifications/unsubscribe-auth/ANDS4FRBLNRVNHLHMYWU5ATRIIWFRANCNFSM4J4BLX4Q

.

--

George W Vandenberghe

IMSG at NOAA/NWS/NCEP/EMC

5830 University Research Ct., Rm. 2141 <https://www.google.com/maps/search/5830+University+Research+Ct

.,+Rm.+2141+%0D%0D%0A+%0D%0D%0A+College+Park,+MD+20740?entry=gmail&source=g>

<https://www.google.com/maps/search/5830+University+Research+Ct

.,+Rm.+2141+%0D%0D%0A+%0D%0D%0A+College+Park,+MD+20740?entry=gmail&source=g>

<https://www.google.com/maps/search/5830+University+Research+Ct

.,+Rm.+2141+%0D%0D%0A+%0D%0D%0A+College+Park,+MD+20740?entry=gmail&source=g>>

College Park, MD 20740 <https://www.google.com/maps/search/5830+University+Research+Ct

.,+Rm.+2141+%0D%0D%0A+%0D%0D%0A+College+Park,+MD+20740?entry=gmail&source=g>

George.Vandenberghe@noaa.gov

301-683-3769(work) 3017751547(cell)

--

George W Vandenberghe

IMSG at NOAA/NWS/NCEP/EMC

5830 University Research Ct., Rm. 2141 <https://www.google.com/maps/search/5830+University+Research+Ct

.,+Rm.+2141+%0D%0D%0A+%0D%0D%0ACollege+Park,+MD+20740?entry=gmail&source=g>

<https://www.google.com/maps/search/5830+University+Research+Ct

.,+Rm.+2141+%0D%0D%0A+%0D%0D%0ACollege+Park,+MD+20740?entry=gmail&source=g>

College Park, MD 20740 <https://www.google.com/maps/search/5830+University+Research+Ct

.,+Rm.+2141+%0D%0D%0A+%0D%0D%0ACollege+Park,+MD+20740?entry=gmail&source=g>

George.Vandenberghe@noaa.gov

301-683-3769(work) 3017751547(cell)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <

https://github.com/NOAA-EMC/global-workflow/issues/5#issuecomment-601480552

, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AKWSMFEHWGEXNUFEKU7LCULRIK4DNANCNFSM4J4BLX4Q

.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub < https://github.com/NOAA-EMC/global-workflow/issues/5#issuecomment-601484194 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ANDS4FROWFG25MOQSENPW2TRIK6K3ANCNFSM4J4BLX4Q

.

--

George W Vandenberghe

IMSG at NOAA/NWS/NCEP/EMC

5830 University Research Ct., Rm. 2141

College Park, MD 20740

George.Vandenberghe@noaa.gov

301-683-3769(work) 3017751547(cell)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/global-workflow/issues/5#issuecomment-601731543, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKWSMFBU76KLWEWJK3LTXL3RIN5DBANCNFSM4J4BLX4Q .

KateFriedman-NOAA commented 4 years ago

Sync merged port2orion with feature/gfsv16b at 1aeda04.

KateFriedman-NOAA commented 4 years ago

Need updated ProdGSI before I can build chgres_recenter_ncio code (some modules come from GSI).

Using a combination of official NCEPLIBS and George V's NCEPLIBS I was able to build:

Prior work got these to build:

Put in request to Hang to get other library versions installed on Orion. Will update all orion modulefiles when official NCEPLIBS is expanded.

CoryMartin-NOAA commented 4 years ago

@KateFriedman-NOAA I have ProdGSI compiled on Orion: /work/noaa/da/Cory.R.Martin/noscrub/git/ProdGSI

branch feature/orion_crm from feature/parallel_ncio added a new modulefile and modified build_all_cmake

I have not tested it to see if it runs, just that the executables build.

KateFriedman-NOAA commented 4 years ago

Great, thanks @CoryMartin-NOAA ! I checked out a copy of feature/orion_crm under my global-workflow port2orion branch and was able to build is successfully. I did notice a few warning messages, can you take a look at my build log?

/work/noaa/nems/kfriedma/git/port2orion/sorc/build_gsi.log

Thanks!

CoryMartin-NOAA commented 4 years ago

@KateFriedman-NOAA I don't have permission to view the file but I also saw a lot of warning messages when compiling. I think it stems from the external GSI utilities and it's complaining whenever subroutines from other modules are not explicitly in "use X, only Y" statements. I think this is an Intel 2020 thing (that's what I used because that is what Hang compiled the libraries in)

RussTreadon-NOAA commented 4 years ago

Same here. I can not cd /work/noaa/nems/kfriedma/git/port2orion/sorc. I do not belong to nems. We need to ensure everyone working on this project has the same access permissions. Please elevate to management.

On Fri, Apr 3, 2020 at 11:21 AM Cory Martin notifications@github.com wrote:

@KateFriedman-NOAA https://github.com/KateFriedman-NOAA I don't have permission to view the file but I also saw a lot of warning messages when compiling. I think it stems from the external GSI utilities and it's complaining whenever subroutines from other modules are not explicitly in "use X, only Y" statements. I think this is an Intel 2020 thing (that's what I used because that is what Hang compiled the libraries in)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/global-workflow/issues/5#issuecomment-608502304, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGNN635SCJP3LU7WOPWDNALRKX5GLANCNFSM4J4BLX4Q .

KateFriedman-NOAA commented 4 years ago

@KateFriedman-NOAA I don't have permission to view the file but I also saw a lot of warning messages when compiling. I think it stems from the external GSI utilities and it's complaining whenever subroutines from other modules are not explicitly in "use X, only Y" statements. I think this is an Intel 2020 thing (that's what I used because that is what Hang compiled the libraries in)

Gotcha, thanks @CoryMartin-NOAA !

@RussTreadon-NOAA Can you read it now? I adjusted my permissions.

RussTreadon-NOAA commented 4 years ago

Yes, I can view build_gsi.log. As Arun said we need to build everything with intel 18. Cory can use intel 2020 for his work. I don't think his branch is intended to be a GFS v16 DA port branch.

On Fri, Apr 3, 2020 at 11:50 AM Kate Friedman notifications@github.com wrote:

@KateFriedman-NOAA https://github.com/KateFriedman-NOAA I don't have permission to view the file but I also saw a lot of warning messages when compiling. I think it stems from the external GSI utilities and it's complaining whenever subroutines from other modules are not explicitly in "use X, only Y" statements. I think this is an Intel 2020 thing (that's what I used because that is what Hang compiled the libraries in)

Gotcha, thanks @CoryMartin-NOAA https://github.com/CoryMartin-NOAA !

@RussTreadon-NOAA https://github.com/RussTreadon-NOAA Can you read it now? I adjusted my permissions.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/global-workflow/issues/5#issuecomment-608517587, or unsubscribe https://github.com/notifications/unsubscribe-auth/AGNN636LASWZYCDRYLYVPJ3RKYATLANCNFSM4J4BLX4Q .

KateFriedman-NOAA commented 4 years ago

Okie dokie...will focus on testing with Hera built execs until a ProdGSI branch that builds with the v16 DA changes is ready. Thanks!

KateFriedman-NOAA commented 4 years ago

Testing with Hera-built execs was mostly successful. Was able to run most of a cycle after resolving environment issues for FV3 model. Currently awaiting library updates on Orion to resolve issues running GSI execs (particularly analcalc and ecen, troubleshooting those jobs currently).

Established pseudo glopara area and established the following subsections:

^ Added those directories to config.base default paths for Orion.

Test GDA (DMPDIR) contains 8 cycles of unrestricted dumps (2020032800 - 2020032918).

The obsproc codes for gfsv16 have been installed, built, and tested on Orion:

Orion-login-2[60] /work/noaa/nems/kfriedma/glopara/git/obsproc/gfsv16$ pwd /work/noaa/nems/kfriedma/glopara/git/obsproc/gfsv16 Orion-login-2[61] /work/noaa/nems/kfriedma/glopara/git/obsproc/gfsv16$ ll | grep -v HERA total 16 drwxr-sr-x 10 kfriedma da 4096 Dec 11 11:13 obsproc_global.iss71402.supportGFSv16/ drwxr-sr-x 7 kfriedma da 4096 Dec 11 11:12 obsproc_prep.iss70457.netcdfhistory_new/

KateFriedman-NOAA commented 4 years ago

Today's Orion users meeting announced they are moving forward with establishing role accounts. Will establish official glopara group space as soon as the account is available to sudo to.

KateFriedman-NOAA commented 4 years ago

Except for GLDAS and WAFS the port2orion branch now builds out of the box on Orion. It uses the following component branches/tags currently:

Opened an issue with GLDAS to get support for Orion: https://github.com/NOAA-EMC/GLDAS/issues/6

GeorgeVandenberghe-NOAA commented 4 years ago

The predecessor to port to hera that I implemented for Hera, August 2019 and Orion December 2019, is obsolete because of numerous repository changes and if I do version 2 it will be redesigned . I would like to at some point get a build system that does not depend on machine name for all of this stuff but I am leaning towards version 2 assuming a supported master build system with ./machine-setup.sh available, set everything there, to make a sane uniform build environment and get rid of application specific module files, all hundred+ of them!

I remember the good old days when there was one compiler, it just worked and /usr/lib, properly and fully maintained by admins, contained all the stuff you needed and when you logged in the system environment was the same for everybody. (although my gripe about "what a command is named this week", dates to the early 80s with primitive unix systems of that time [ and was also lifted from a Datamation article in 1983} )

On Mon, May 11, 2020 at 12:52 PM Kate Friedman notifications@github.com wrote:

Except for GLDAS and WAFS the port2orion branch now builds out of the box on Orion. It uses the following component branches currently:

- https://github.com/DusanJovic-NOAA/ufs-weather-model/tree/orion_gfs.v16

Opened an issue with GLDAS to get support for Orion: NOAA-EMC/GLDAS#6 https://github.com/NOAA-EMC/GLDAS/issues/6

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/global-workflow/issues/5#issuecomment-626822597, or unsubscribe https://github.com/notifications/unsubscribe-auth/ANDS4FQ4HCYKK2HLLQMY3KLRRAUNLANCNFSM4J4BLX4Q .

--

George W Vandenberghe

IMSG at NOAA/NWS/NCEP/EMC

5830 University Research Ct., Rm. 2141

College Park, MD 20740

George.Vandenberghe@noaa.gov

301-683-3769(work) 3017751547(cell)

KateFriedman-NOAA commented 4 years ago

Committed at 206797c: New GLDAS tag which includes Orion support. New rocoto module (rocoto/1.3.2) available on Orion which includes updates to fix UNAVAILABLE job states on SLURM machines. Updates to UFS_UTILS for Orion...will test and confirm to George Gayno in https://github.com/NOAA-EMC/UFS_UTILS/issues/26.

KateFriedman-NOAA commented 4 years ago

Sync merged release/gfsv16.0.0 branch into port2orion. Now testing port2orion on Orion, Hera, and WCOSS-Dell for several resolution combinations before prepping to merge back to develop.