Solve zlib issue for NetCDF/libpng, fix static build of netCDF, add install instructions for Stampede, Orion, Ubuntu Linux

climbfuji commented 4 years ago

For a discussion arount the libz issue, see here: https://github.com/NOAA-EMC/NCEPLIBS-external/issues/28. The solution/workaround for the release is to request that either both NetCDF and libpng are installed as part of NCEPLIBS-externals, or neither of them. This PR adds a guard to CMakeLists.txt to enforce this and also adds information to the top-level README.md.

In addition, this PR contains:

install instructions for Stampede+Intel, Orion+Intel, generic Ubuntu+GNU
update of cmake submodule pointer with changes from last week (all tested).
static build of NetCDF (tested with downstream NCEPLIBS and ufs-weather-model build.sh)
add LGPL license

ceceliadid commented 4 years ago

Hi @climbfuji I was able to build nceplibs and nceplibs-external and the WM on Stampede2 using the README_stampede_intel. It all took less than an hour, which is great! Few comments:

In the building WM part of the file, you still have your work dir in the command instead of $WORK: . /work/06146/tg854455/stampede2/NCEPLIBS-ufs-v1.0.0.beta03/bin/setenv_nceplibs.sh
Maybe dumb, but for students it wouldn't hurt to tell them to set WORK before starting the sequence.
Again, maybe dumb, but it might be good to add a sentence like "If you already ran the module load commands and set the compilers and NETCDF variables before you built NCEPLIBS and NCEPLIBS-external, you don't need to do it again" when they're listed again under building the WM.
Do you need to set the COMPILER variable before building the WM? The WM UG says you need to, but it's not included here. Should it be?

climbfuji commented 4 years ago

Hi @climbfuji I was able to build nceplibs and nceplibs-external and the WM on Stampede2 using the README_stampede_intel. It all took less than an hour, which is great! Few comments:

In the building WM part of the file, you still have your work dir in the command instead of $WORK: . /work/06146/tg854455/stampede2/NCEPLIBS-ufs-v1.0.0.beta03/bin/setenv_nceplibs.sh

Maybe dumb, but for students it wouldn't hurt to tell them to set WORK before starting the sequence.

I should change this to $WORK/...

Again, maybe dumb, but it might be good to add a sentence like "If you already ran the module load commands and set the compilers and NETCDF variables before you built NCEPLIBS and NCEPLIBS-external, you don't need to do it again" when they're listed again under building the WM.

It doesn't hurt at all doing it twice.

Do you need to set the COMPILER variable before building the WM? The WM UG says you need to, but it's not included here. Should it be?

No, I think this is legacy. You need to set the cmake platform environment variable, which includes the COMPILER part.

climbfuji commented 4 years ago

I forgot to say, the environment variable WORK is automatically defined for the user (by the system), as far as I remember. I fixed the first issue (hard-coded path to my install dir) in the instructions.

climbfuji commented 4 years ago

@mark-a-potts @kgerheiser this version is ready to merge. I tested it to exhaustion on gaea for the static build of netCDF, and on the following platforms for the standard/dynamic builds: hera, jet, orion, macOS, redhat, ubuntu. After merging, the release branch can be tagged as beta03, corresponding to the beta03 tag of NCEPLIBS. Thanks!

ceceliadid commented 4 years ago

Hi @climbfuji when you fixed the hard coded path to your install dir I think there was a cut&paste error: . $WORK/NCEPLIBS-ufs-v1.0.0.beta04/NCEPLIBS-ufs-v1.0.0.beta04/bin/setenv_nceplibs.sh

Looks like it has an extra NCEPLIBS-ufs-v1.0.0.beta04 in it.

mark-a-potts commented 4 years ago

It does look that way, but I don't have an account on Stampede to check it out.

-M

On 3/1/20 11:55 PM, ceceliadid wrote:

Hi @climbfuji https://github.com/climbfuji when you fixed the hard coded path to your install dir I think there was a cut&paste error: . $WORK/NCEPLIBS-ufs-v1.0.0.beta04/NCEPLIBS-ufs-v1.0.0.beta04/bin/setenv_nceplibs.sh

Looks like it has an extra NCEPLIBS-ufs-v1.0.0.beta04 in it.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/NCEPLIBS-external/pull/29?email_source=notifications&email_token=AH4Q2UVWJJBR3FYGEWXGROLRFM33NA5CNFSM4K2UYDEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENN42OI#issuecomment-593218873, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH4Q2UR33D4GKN4POUYPYMDRFM33NANCNFSM4K2UYDEA.

-- Mark A. Potts, Ph.D. Sr. HPC Software Developer RedLine Performance Solutions, LLC Phone 202-744-9469 Mark.Potts@noaa.gov mpotts@redlineperf.com

ceceliadid commented 4 years ago

@mark-a-potts On Stampede if I use what's currently in the file I get login2(1006)$ -bash: /work/07253/tg865527/stampede2/NCEPLIBS-ufs-v1.0.0.beta04/NCEPLIBS-ufs-v1.0.0.beta04/bin/setenv_nceplibs.sh: No such file or directory

and if I fix it I get: login2(1007)$ login2(1004)$ . $WORK/NCEPLIBS-ufs-v1.0.0.beta04/bin/setenv_nceplibs.sh Setting environment variables for NCEPLIBS and its dependencies

If you need confirmation while Dom is gone maybe ask @rsdunlapiv or @jedwards4b

mark-a-potts commented 4 years ago

That is good enough for me. I'll create a PR and fix it.

-M

On 3/2/20 9:54 AM, ceceliadid wrote:

@mark-a-potts https://github.com/mark-a-potts On Stampede if I use what's currently in the file I get login2(1006)$ -bash: /work/07253/tg865527/stampede2/NCEPLIBS-ufs-v1.0.0.beta04/NCEPLIBS-ufs-v1.0.0.beta04/bin/setenv_nceplibs.sh: No such file or directory

and if I fix it I get: login2(1007)$ login2(1004)$ . $WORK/NCEPLIBS-ufs-v1.0.0.beta04/bin/setenv_nceplibs.sh Setting environment variables for NCEPLIBS and its dependencies

If you need confirmation while Dom is gone maybe ask @rsdunlapiv https://github.com/rsdunlapiv or @jedwards4b https://github.com/jedwards4b

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/NCEPLIBS-external/pull/29?email_source=notifications&email_token=AH4Q2URKAJS5XWYA7PA6TB3RFPCA3A5CNFSM4K2UYDEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENPTC7Y#issuecomment-593441151, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH4Q2UU6PZJLPIAXFH736TDRFPCA3ANCNFSM4K2UYDEA.

-- Mark A. Potts, Ph.D. Sr. HPC Software Developer RedLine Performance Solutions, LLC Phone 202-744-9469 Mark.Potts@noaa.gov mpotts@redlineperf.com

mark-a-potts commented 4 years ago

I have updated the file and re-tagged for ufs-v1.0.0.beta04.

-M

On 3/2/20 9:54 AM, ceceliadid wrote:

@mark-a-potts https://github.com/mark-a-potts On Stampede if I use what's currently in the file I get login2(1006)$ -bash: /work/07253/tg865527/stampede2/NCEPLIBS-ufs-v1.0.0.beta04/NCEPLIBS-ufs-v1.0.0.beta04/bin/setenv_nceplibs.sh: No such file or directory

and if I fix it I get: login2(1007)$ login2(1004)$ . $WORK/NCEPLIBS-ufs-v1.0.0.beta04/bin/setenv_nceplibs.sh Setting environment variables for NCEPLIBS and its dependencies

If you need confirmation while Dom is gone maybe ask @rsdunlapiv https://github.com/rsdunlapiv or @jedwards4b https://github.com/jedwards4b

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NOAA-EMC/NCEPLIBS-external/pull/29?email_source=notifications&email_token=AH4Q2URKAJS5XWYA7PA6TB3RFPCA3A5CNFSM4K2UYDEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENPTC7Y#issuecomment-593441151, or unsubscribe https://github.com/notifications/unsubscribe-auth/AH4Q2UU6PZJLPIAXFH736TDRFPCA3ANCNFSM4K2UYDEA.

-- Mark A. Potts, Ph.D. Sr. HPC Software Developer RedLine Performance Solutions, LLC Phone 202-744-9469 Mark.Potts@noaa.gov mpotts@redlineperf.com

climbfuji commented 4 years ago

Sorry for this mishap, and thanks for fixing it!

NOAA-EMC / NCEPLIBS-external

Solve zlib issue for NetCDF/libpng, fix static build of netCDF, add install instructions for Stampede, Orion, Ubuntu Linux #29