NCAR / DART

Data Assimilation Research Testbed
https://dart.ucar.edu/
Apache License 2.0
194 stars 145 forks source link

mkmf.template.intel.osx out of date #219

Closed hkershaw-brown closed 3 years ago

hkershaw-brown commented 3 years ago

:bug: Your bug may already be reported! Please search on the issue tracker before creating a new issue.

Describe the bug

  1. List the steps someone needs to take to reproduce the bug.
    run quickbuild.csh with mkmf.template.intel.osx for mkmf.template

  2. What was the expected outcome? Compile DART executables without error

  3. What actually happened?
    For the production options, there is a remark ifort: command line remark #10148: option '-vec-report0' not supported For the debug options, the compilation errors out: fortcom: Error: Unrecognized keyword "argument_checking" for option "-warn argument_checking"

Error Message

fortcom: Error: Unrecognized keyword "argument_checking" for option "-warn argument_checking

Which model(s) are you working with?

Version of DART

v9.10.3

Have you modified the DART code?

No

Build information

Please describe:

  1. macOS Catelina 10.15.7
  2. intel
hkershaw-brown commented 3 years ago

compilation errors also for the following

assimilation_code/programs/obs_total_error/work/input.nml obs_type~s~_files models/model_mod_tools/work/path_names_test_quad_irreg_interp. missing options_mod models/model_mod_tools/work/path_names_test_quad_reg_interp. missing options_mod

hkershaw-brown commented 3 years ago

Preprocess error with intel 2021 mac: (one API) ifort (IFORT) 2021.2.0 20210228

 ./preprocess 
forrtl: severe (257): formatted I/O to unit open for unformatted transfers, unit 10, file /Users/hkershaw/DART/pull_requests/pull_215/models/lorenz_96/work/input.nml
Image              PC                Routine            Line        Source             
libnetcdff.7.dyli  0000000109817D4B  Unknown               Unknown  Unknown
preprocess         0000000106493CAE  Unknown               Unknown  Unknown
preprocess         000000010643A0BB  _utilities_mod_mp         425  utilities_mod.f90
preprocess         0000000106437F3A  _utilities_mod_mp         204  utilities_mod.f90
preprocess         000000010645B8BB  _MAIN__                   195  preprocess.f90
preprocess         000000010642033E  Unknown               Unknown  Unknown

Error does not occur on Cheyenne ifort (IFORT) 2021.2.0 20210228

hkershaw-brown commented 3 years ago

This might be old netcdf + new intel:

hkershaw@cisl-fisher intel2021 $ cat hello.f90 
program hello_world

implicit none

print*, "Hello Helen"

end program 

hkershaw@cisl-fisher intel2021 $ ifort -L/Users/hkershaw/software/intel/lib -lnetcdff -lnetcdf hello.f90 
hkershaw@cisl-fisher intel2021 $ ./a.out 
forrtl: severe (257): formatted I/O to unit open for unformatted transfers, unit -1, file /dev/ttys014
Image              PC                Routine            Line        Source             
libnetcdff.7.dyli  0000000106CA1D4B  for__io_return        Unknown  Unknown
a.out              0000000106BC86AC  for_write_seq_lis     Unknown  Unknown
a.out              0000000106BC75B6  MAIN__                Unknown  Unknown
a.out              0000000106BC751E  main                  Unknown  Unknown
hkershaw@cisl-fisher intel2021 $ ifort hello.f90 
hkershaw@cisl-fisher intel2021 $ ./a.out 
 Hello Helen
hkershaw-brown commented 3 years ago

compilation errors also for the following

assimilation_code/programs/obs_total_error/work/input.nml obs_type~s~_files models/model_mod_tools/work/path_names_test_quad_irreg_interp. missing options_mod models/model_mod_tools/work/path_names_test_quad_reg_interp. missing options_mod

branch build_fixes has the fixes for these

hkershaw-brown commented 3 years ago

For compiling openmpi and netcdf with intel on mac, ifort does not recognize libtool option -force_load so you have to change the configure script, or set lt_cv_ld_force_load=no when you configure.

lt_cv_ld_force_load=no CC=icc FC=ifort CXX=icpc .... ./configure --prefix=____

openmpi 4.0.4 netcdf-c-4.8.0 netcdf-fortran-4.5.3 ifort (IFORT) 2021.2.0 20210228

an example of the errors:

Undefined symbols for architecture x86_64:
  "_ompi_buffer_detach_f08", referenced from:
hkershaw-brown commented 3 years ago

pgi 20.4 Cheyenne. I think this is the \

pgf90 -O -I/glade/u/apps/ch/opt/netcdf/4.7.4/pgi/20.4//include  -c  ../../../assimilation_code/modules/utilities/parse_args_mod.f90
PGF90-S-0026-Unmatched quote (../../../assimilation_code/modules/utilities/parse_args_mod.f90: 138)
  0 inform,   0 warnings,   1 severes, 0 fatal for get_args_from_string
nancycollins commented 3 years ago

try making line 138?

if (thisc == '\\') then

(compare to '\\' -- two consecutive backslashes. that works for me on ifort on my mac. )

edit: dang smart parsing in this comment.

hkershaw-brown commented 3 years ago

does the parse_args_mod still work with //? pgi has -Mbackslash which "instructs the compiler to treat the backslash as a normal character, and not as an escape character in quoted strings".

nancycollins commented 3 years ago

sadly, no. for now i think the compiler flag may be a better fix but it would be nice not to have to count on that. there may be a different way to make a character constant that it can compare against instead of the literal string '\' .

at least i don't believe it's working. this is hard to test because so many parts of the system want to help you out with escaped characters. if running interactively, the shell reads the line and parses it before the program sees it, so it parses \\ into \ . if you run the program with stdin redirected to a file i don't know what the fortran 'read next line' routine does with quotes and escapes before it gets put into a character string variable.

in the developer test directory, under utilities is a parse_args_test.in input file but it's very simplistic. also, the parse_args_test program is a bit broken in that it has 2 sections - one for testing white-space separated words from a line, and one for name=value pairs. but it wants control-d to end the first part of the test and move to the second. control-d stays in the input stream and ends both parts of the test. i will open an issue with making parse_args_test and the input file for the test more robust.

hkershaw-brown commented 3 years ago

https://www.explainxkcd.com/wiki/index.php/1638:_Backslashes

hkershaw-brown commented 3 years ago

pgi does not like the inflation flavors being integers.

 ERROR FROM:
  source : utilities_mod.f90
  routine: check_namelist_read
  message:  INVALID NAMELIST ENTRY:    inf_flavor                  = 2,                       0, in namelist filter_nml
- inf_flavor                  = 2,                       0,
+ inf_flavor                 = '2',                     '0',
nancycollins commented 3 years ago

for the backslash issue here is a small test program. i think using ACHAR() to set a character string of length 1 is the best solution here. i can make a branch and fix it if you want and beef up the parse tests at the same time. then you don't have to fool with compiler flags and it should be more portable. let me know. there may be other much higher priorities.

in spite of the display, there are 2 backslashes in the BS1 variable. if you cut and paste this they will show up. i don't know how to turn off markdown in these comments.

cat > bs_test.f90 <<'EOF' ! this is a test to see which compilers support ! using double backslashes to initialize a single ! character constant as ASCII code 92 (octal 134, ! hex 5C). The most portable way seems to be to ! use the ACHAR intrinsic to set the string directly ! from the ASCII code.

program bs_test

! if this program does not compile, comment out this next line ! and the two lines indicated further below. character(len=1), parameter :: BS1 = '\'

character(len=1) :: BS2

character(len=20) :: bob

! this is a fortran intrinsic and should be portable BS2 = achar(92)

bob = 'this is a string' print *, 'initial val: ', trim(bob)

! if BS1 give errors comment out these 2 lines bob(10:10) = BS1(1:1) print *, 'altered1:', trim(bob)

! this should be portable bob(6:6) = BS2(1:1) print *, 'altered2:', bob

print print , 'success for style 1 is if the program compiles' print , 'and the output replaces the space between' print , 'a and string with a backslash.' print print , 'success for style 2 is if the program compiles' print *, 'and replaces the i in is with a backslash.'

end program 'EOF'

hkershaw-brown commented 3 years ago

Just so I am following, for get_args_from_string the \ is to ignore \n \r \whatever in the string.
So you can't have backslashes in your strings, e.g. unitsof\doom or windows\path

131    ! next character on line
132    thisc = argline(thisoff:thisoff)
133 
134 if (debug) print *, 'thisoff, finaloff, inword, endword, thisc = ', thisoff, finaloff, &
135                      inword, '"'//endword//'"', ' ', '"'//thisc//'"'
136 
137    ! escaped chars - backslash prevents interpretation of next char
138    if (thisc == '\') then
139       ! move the remainder of the string over, overwriting the \ and
140       ! skipping the next char.
141       do i=thisoff, finaloff-1
142          argline(i:i) = argline(i+1:i+1)
143       enddo
144       argline(finaloff:finaloff) = ' '
145       finaloff = finaloff-1
146       thisoff = thisoff+1
147       cycle NEXTCHAR
148    endif

And the problem is '\' is being treated by pgi like C where \ is an escape character

So the code would become the following using achar(92)

-138    if (thisc == '\') then
+138    if (thisc == achar(92)) then

protip:

use 
  ``` for \\ ``` 
nancycollins commented 3 years ago

yes, the intent in get_args_from_string() is that in the input string backslash skips interpretation of the next char, whatever it is. so backslash followed by a newline or carriage return would put those chars into the output string. you could follow backslash with a quote to put the quote into the string instead of it being interpreted as the end of string. backslash followed by backslash (double backslash) in your input string would become a single backslash in the output.

yes, pgi is treating the backslash in the source code like C, so it's turning quote backslash quote into 'quote to start a string, quote as the first character of that string' then erroring out because it thinks it is missing the closing quote.

yes to how to use achar, although i might put it in a variable because if you search for double backslash (good luck) you will find several other places this occurs in the file.

edit: updated to try to be more clear

nancycollins commented 3 years ago

one more thought about get_args_from_string() - this routine takes a character string as input. the contents of that string had to be created by something. it could be a character constant in another program, it could be read from the terminal, it could be a variable from a namelist. whatever is reading the characters and putting them into the string before calling this routine may also try to interpret special characters like backslash and quotes. so this is tricky to test.

hkershaw-brown commented 3 years ago

note - got a dart support email about the backslash problem with PGI

hkershaw-brown commented 2 years ago

nvfortran also complains about \

NVFORTRAN-S-0026-Unmatched quote  (../../../assimilation_code/modules/utilities/parse_args_mod.f90: 138)
  0 inform,   0 warnings,   1 severes, 0 fatal for get_args_from_string
nancycollins commented 1 year ago

in re-reading this thread i am reminded that ACHAR is a way to set a character variable without having an explicit backslash in the code and having to fool with compiler flags. i think the fix for this and issue #555 should be:

character(len=1), parameter :: BACKSLASH = ACHAR(92)

...

if (thisc == BACKSLASH) then