search

CEMPD / SMOKE

Create emissions inputs for multiple air quality modeling systems with unmatched speed and flexibility
https://www.cmascenter.org/smoke/
45 stars 21 forks source link

notes on SMOKE Installation Instructions #89

Open cseppan opened 5 months ago

cseppan commented 5 months ago

Tested on base installation of Ubuntu 22.04 LTS. Used gcc and gfortran, these were installed already.

Needed to install tcsh (sudo apt install tcsh), maybe list this is a prerequisite

Could add an example of setting MASTERDIR

setenv MASTERDIR /path/to/install/dir [tcsh] export MASTERDIR=/path/to/install/dir [bash]

Did IOAPI download, then jumped down to installations instructions for required packages

After creating the various compile.sh scripts, each needs to be set as executable

chmod u+x $MASTERDIR/zlib-1.3/compile.sh chmod u+x $MASTERDIR/szip-2.1.1/compile.sh chmod u+x $MASTERDIR/hdf5-1.14.3/compile.sh chmod u+x $MASTERDIR/netcdf-c-4.9.2/compile.sh chmod u+x $MASTERDIR/netcdf-fortran-4.6.1/compile.sh

No problems until netcdf-fortran. Running

$MASTERDIR/netcdf-fortran-4.6.1/compile.sh configure

failed with

checking for netcdf.h... no
configure: error: netcdf.h could not be found. Please set CPPFLAGS.

I confirmed that $MASTERDIR/ioapi-3.2/dependencies/netcdfc.4.9.2_gcc/include/ has netcdf.h, but it doesn't look like the wrapper script is telling configure about this location

cseppan commented 5 months ago

Note about netcdf.h - on the longleaf compute nodes, there's an older version of netcdf.h in /usr/include/ so the netcdf-fortran configure finds and uses that

cseppan commented 5 months ago

A few more notes about trying to compile using gcc/gfortran on longleaf

Running compile.ioapi.csh, the IOAPI library (Linux2_x86_64/libioapi.a) compiled successfully but the m3tools failed. End of compile.Linux2_x86_64.log:

make[1]: Entering directory '/work/users/c/s/cseppan/smoke_install/ioapi-3.2/m3tools'
cd /work/users/c/s/cseppan/smoke_install/ioapi-3.2/Linux2_x86_64; gfortran -I/work/users/c/s/cseppan/smoke_install/ioapi-3.2/ioapi -I/work/users/c/s/cseppan/smoke_install/ioapi-3.2/Linux2_x86_64 -DIOAPI_NCF4 -DAUTO_ARRAYS=1 -DF90=1 -DFLDMN=1 -DFSTR_L=int -DIOAPI_NO_STDOUT=1 -DNEED_ARGS=1  -O3 -std=legacy -ffast-math -funroll-loops -m64 -static  -DAUTO_ARRAYS=1 -DF90=1 -DFLDMN=1 -DFSTR_L=int -DIOAPI_NO_STDOUT=1 -DNEED_ARGS=1 -c /work/users/c/s/cseppan/smoke_install/ioapi-3.2/m3tools/airs2m3.f
cd /work/users/c/s/cseppan/smoke_install/ioapi-3.2/Linux2_x86_64; gfortran  airs2m3.o -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/Linux2_x86_64 -lioapi -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/dependencies/netcdff.4.6.1_gcc/lib -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/dependencies/netcdfc.4.9.2_gcc/lib -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/dependencies/hdf5.1.14.3_gcc/lib -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/dependencies/szip.2.1.1_gcc/lib -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/dependencies/zlib.1.3_gcc/lib -lnetcdff -lnetcdf -lhdf5_hl -lhdf5 -lm -lz -lsz -ldl -fopenmp -static -L/usr/lib64 -lm -lpthread -lc  -o airs2m3
/usr/bin/ld: cannot find -lgfortran
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:248: airs2m3] Error 1
make[1]: Leaving directory '/work/users/c/s/cseppan/smoke_install/ioapi-3.2/m3tools'
make: *** [Makefile:205: all] Error 2

Since the library was created, I continued on to SMOKE. Before running $MASTERDIR/SMOKE_v5.0/compile.SMOKE.csh you need to be in the directory $MASTERDIR/SMOKE_v5.0/. This is true of the other compile.sh scripts (for the dependencies) but all of those have cd commands included in the instructions.

The SMOKE compile failed at the same linking stage as m3tools

cd /work/users/c/s/cseppan/smoke_install/SMOKE_v5.0/Linux2_x86_64; gfortran -I/work/users/c/s/cseppan/smoke_install/ioapi-3.2/ioapi/fixed_src -I/work/users/c/s/cseppan/smoke_install/SMOKE_v5.0/src/inc -I/work/users/c/s/cseppan/smoke_install/ioapi-3.2/Linux2_x86_64                 -ffixed-line-length-132  -fno-backslash -fcheck=all -Wall -march=native -ffast-math -funroll-loops         -O3 -std=legacy -ffast-math -funroll-loops -m64 -static  -DAUTO_ARRAYS=1 -DF90=1 -DFLDMN=1 -DFSTR_L=int -DIOAPI_NO_STDOUT=1 -DNEED_ARGS=1 -o normbeis3 normbeis3.o rdb3fac.o rdb4fac.o rdb4fac_csv.o normbeis314.o normbeis361.o normbeis370.o libemmod.a libsmoke.a libfileset.a -L/work/users/c/s/cseppan/smoke_install/SMOKE_v5.0/Linux2_x86_64 -lfileset -lsmoke -lemmod -lfileset -lsmoke -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/Linux2_x86_64 -lioapi -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/dependencies/netcdff.4.6.1_gcc/lib -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/dependencies/netcdfc.4.9.2_gcc/lib -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/dependencies/hdf5.1.14.3_gcc/lib -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/dependencies/szip.2.1.1_gcc/lib -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/dependencies/zlib.1.3_gcc/lib -lnetcdff -lnetcdf -lhdf5_hl -lhdf5 -lm -lz -lsz -ldl -fopenmp -static -L/usr/lib64 -lm -lpthread -lc 
/usr/bin/ld: cannot find -lgfortran
collect2: error: ld returned 1 exit status
make: *** [Makefile:687: normbeis3] Error 1
hnqtran commented 5 months ago

Note about netcdf.h - on the longleaf compute nodes, there's an older version of netcdf.h in /usr/include/ so the netcdf-fortran configure finds and uses that

Thanks for the hint. I added setenv CPPFLAGS "-I${NETCDFC}/include" in compile.csh script for netcdf-fortran to tell the compiler where to look for netcdf.h

hnqtran commented 5 months ago

A few more notes about trying to compile using gcc/gfortran on longleaf

Running compile.ioapi.csh, the IOAPI library (Linux2_x86_64/libioapi.a) compiled successfully but the m3tools failed. End of compile.Linux2_x86_64.log:

make[1]: Entering directory '/work/users/c/s/cseppan/smoke_install/ioapi-3.2/m3tools'
cd /work/users/c/s/cseppan/smoke_install/ioapi-3.2/Linux2_x86_64; gfortran -I/work/users/c/s/cseppan/smoke_install/ioapi-3.2/ioapi -I/work/users/c/s/cseppan/smoke_install/ioapi-3.2/Linux2_x86_64 -DIOAPI_NCF4 -DAUTO_ARRAYS=1 -DF90=1 -DFLDMN=1 -DFSTR_L=int -DIOAPI_NO_STDOUT=1 -DNEED_ARGS=1  -O3 -std=legacy -ffast-math -funroll-loops -m64 -static  -DAUTO_ARRAYS=1 -DF90=1 -DFLDMN=1 -DFSTR_L=int -DIOAPI_NO_STDOUT=1 -DNEED_ARGS=1 -c /work/users/c/s/cseppan/smoke_install/ioapi-3.2/m3tools/airs2m3.f
cd /work/users/c/s/cseppan/smoke_install/ioapi-3.2/Linux2_x86_64; gfortran  airs2m3.o -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/Linux2_x86_64 -lioapi -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/dependencies/netcdff.4.6.1_gcc/lib -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/dependencies/netcdfc.4.9.2_gcc/lib -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/dependencies/hdf5.1.14.3_gcc/lib -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/dependencies/szip.2.1.1_gcc/lib -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/dependencies/zlib.1.3_gcc/lib -lnetcdff -lnetcdf -lhdf5_hl -lhdf5 -lm -lz -lsz -ldl -fopenmp -static -L/usr/lib64 -lm -lpthread -lc  -o airs2m3
/usr/bin/ld: cannot find -lgfortran
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:248: airs2m3] Error 1
make[1]: Leaving directory '/work/users/c/s/cseppan/smoke_install/ioapi-3.2/m3tools'
make: *** [Makefile:205: all] Error 2

Since the library was created, I continued on to SMOKE. Before running $MASTERDIR/SMOKE_v5.0/compile.SMOKE.csh you need to be in the directory $MASTERDIR/SMOKE_v5.0/. This is true of the other compile.sh scripts (for the dependencies) but all of those have cd commands included in the instructions.

The SMOKE compile failed at the same linking stage as m3tools

cd /work/users/c/s/cseppan/smoke_install/SMOKE_v5.0/Linux2_x86_64; gfortran -I/work/users/c/s/cseppan/smoke_install/ioapi-3.2/ioapi/fixed_src -I/work/users/c/s/cseppan/smoke_install/SMOKE_v5.0/src/inc -I/work/users/c/s/cseppan/smoke_install/ioapi-3.2/Linux2_x86_64                 -ffixed-line-length-132  -fno-backslash -fcheck=all -Wall -march=native -ffast-math -funroll-loops         -O3 -std=legacy -ffast-math -funroll-loops -m64 -static  -DAUTO_ARRAYS=1 -DF90=1 -DFLDMN=1 -DFSTR_L=int -DIOAPI_NO_STDOUT=1 -DNEED_ARGS=1 -o normbeis3 normbeis3.o rdb3fac.o rdb4fac.o rdb4fac_csv.o normbeis314.o normbeis361.o normbeis370.o libemmod.a libsmoke.a libfileset.a -L/work/users/c/s/cseppan/smoke_install/SMOKE_v5.0/Linux2_x86_64 -lfileset -lsmoke -lemmod -lfileset -lsmoke -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/Linux2_x86_64 -lioapi -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/dependencies/netcdff.4.6.1_gcc/lib -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/dependencies/netcdfc.4.9.2_gcc/lib -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/dependencies/hdf5.1.14.3_gcc/lib -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/dependencies/szip.2.1.1_gcc/lib -L/work/users/c/s/cseppan/smoke_install/ioapi-3.2/dependencies/zlib.1.3_gcc/lib -lnetcdff -lnetcdf -lhdf5_hl -lhdf5 -lm -lz -lsz -ldl -fopenmp -static -L/usr/lib64 -lm -lpthread -lc 
/usr/bin/ld: cannot find -lgfortran
collect2: error: ld returned 1 exit status
make: *** [Makefile:687: normbeis3] Error 1

I could not replicate this error either on longleaf or on Ubuntu virtual machine. Maybe you were using different version of gcc for compiling netcdf than for compiling ioapi? gcc is quite strict on requirement of same version of gcc/gfortran across all software package.

Btw I test the whole setup with gfortran on Ubuntu virtual machine and it was able to compiled all packages including SMOKE and dependencies.

cseppan commented 4 months ago

An updated version of zlib was released and the download link in the compilation instructions is no longer available. The new package is https://www.zlib.net/zlib-1.3.1.tar.gz (old https://www.zlib.net/zlib-1.3.tar.gz). This new release extracts into a directory named zlib-1.3.1, so various commands in the instructions need a minor update.

cseppan commented 4 months ago

By the way, I figured out the linker problem that was causing this error:

/usr/bin/ld: cannot find -lgfortran
collect2: error: ld returned 1 exit status
make: *** [Makefile:687: normbeis3] Error 1

If you don't have a gcc module loaded on the UNC longleaf system, there's a default version of gcc and gfortran available. However, the libraries aren't set up for use, so while you can compile individual code files, linking to the required libgfortran library will fail. I needed to load the gcc module (I used version 11.2.0) and make sure to save it to my default module set so that the shell scripts will have a full working gcc installation when they run. This problem is specific to how the longleaf cluster is set up, so not generally applicable.