Installation errors with gcc 10

[ashwinvis]

With gcc 10.3.0, I get

mpic++   -c -DDEBUG=0  -O3    -o NETCDF_IO/add_attr_to_file.o NETCDF_IO/add_attr_to_file.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
NETCDF_IO/add_attr_to_file.cpp: In function 'void add_attr_to_file(const char*, double, const char*, MPI_Comm)':
NETCDF_IO/add_attr_to_file.cpp:23:38: error: format not a string literal and no format arguments [-Werror=format-security]
   23 |         snprintf(buffer, 50, filename);
      |                                      ^

If I do CFLAGS := -w to suppress warnings, it goes a bit further, but errors again

mpic++   -c -DDEBUG=0 -w -O3    -o NETCDF_IO/add_attr_to_file.o NETCDF_IO/add_attr_to_file.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -O3    -o NETCDF_IO/add_var_to_file.o NETCDF_IO/add_var_to_file.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -O3    -o NETCDF_IO/check_file_existence.o NETCDF_IO/check_file_existence.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -O3    -o NETCDF_IO/initialize_output_file.o NETCDF_IO/initialize_output_file.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -O3    -o NETCDF_IO/initialize_particle_file.o NETCDF_IO/initialize_particle_file.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -O3    -o NETCDF_IO/initialize_postprocess_file.o NETCDF_IO/initialize_postprocess_file.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -O3    -o NETCDF_IO/initialize_projected_particle_file.o NETCDF_IO/initialize_projected_particle_file.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -O3    -o NETCDF_IO/initialize_regions_file.o NETCDF_IO/initialize_regions_file.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -O3    -o NETCDF_IO/initialize_subset_file.o NETCDF_IO/initialize_subset_file.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -O3    -o NETCDF_IO/load_region_definitions.o NETCDF_IO/load_region_definitions.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -O3    -o NETCDF_IO/nc_err.o NETCDF_IO/nc_err.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -O3    -o NETCDF_IO/package_field.o NETCDF_IO/package_field.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
NETCDF_IO/package_field.cpp: In function 'void package_field(std::vector<short int>&, double&, double&, const std::vector<double>&, const std::vector<bool>*, MPI_Comm)':
NETCDF_IO/package_field.cpp:59:54: error: 'fmiddle' not specified in encerrorlosing 'parallel'
   59 |                 local_double = (original.at(index) - fmiddle) / frange;
      |                                                      ^~~~~~~
NETCDF_IO/package_field.cpp:49:13: error: enclosing 'parallel'
   49 |     #pragma omp parallel \
      |             ^~~
NETCDF_IO/package_field.cpp:59:65: error: 'frange' not specified in enclosing 'parallel'
   59 |                 local_double = (original.at(index) - fmiddle) / frange;
      |                                                                 ^~~~~~
NETCDF_IO/package_field.cpp:49:13: error: enclosing 'parallel'
   49 |     #pragma omp parallel \
      |             ^~~
make: *** [Makefile:78: NETCDF_IO/package_field.o] Error 1

[bastorer]

Oh, it looks like gcc wants consts to be specified as openmp-shared (whereas intel auto-imposes shared consts).

Thanks! I've mostly been compiling with intel lately, but will rebuild a gcc version to hunt for other syntax mis-matches.

[ashwinvis]

Great. Let me know when you fixed it for gcc

[bastorer]

Ah, okay, it looks like GCC 9 and onwards changed how const-declared variables are handle in OpenMP clauses (https://gcc.gnu.org/gcc-9/porting_to.html) in a way the lightly breaks back-compatibility. I'm resolving the (shockingly many) warnings that pop up with GCC (but not intel, weirdly enough), but resolving the OpenMP predetermined errors shouldn't be too bad. It looks like it only hits small variables, so setting them as firstprivate should do it. I'm testing on GCCv7 right now, but will do a 9+ test afterwards (just need to see which of the servers have a GCCv9+ installation of the modules...).

Will hopefully be done later today.

[bastorer]

@SalahKouhen tagging you here so you get the updates when this is resolved too

[bastorer]

I've updated the code base and it now compiles for me without warnings or errors with GCCv8 and GCCv9 ( I didn't have access to a GCCv10 module set that had the libraries that I needed, but v9 is on the same side of the openmp behaviour change so with a little luck v10 should work as well )

[ashwinvis]

It goes a bit further now, but errors in a different file

mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o NETCDF_IO/add_attr_to_file.o NETCDF_IO/add_attr_to_file.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o NETCDF_IO/add_var_to_file.o NETCDF_IO/add_var_to_file.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o NETCDF_IO/check_file_existence.o NETCDF_IO/check_file_existence.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o NETCDF_IO/initialize_output_file.o NETCDF_IO/initialize_output_file.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o NETCDF_IO/initialize_particle_file.o NETCDF_IO/initialize_particle_file.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o NETCDF_IO/initialize_postprocess_file.o NETCDF_IO/initialize_postprocess_file.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o NETCDF_IO/initialize_projected_particle_file.o NETCDF_IO/initialize_projected_particle_file.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o NETCDF_IO/initialize_regions_file.o NETCDF_IO/initialize_regions_file.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o NETCDF_IO/initialize_subset_file.o NETCDF_IO/initialize_subset_file.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o NETCDF_IO/load_region_definitions.o NETCDF_IO/load_region_definitions.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o NETCDF_IO/nc_err.o NETCDF_IO/nc_err.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o NETCDF_IO/package_field.o NETCDF_IO/package_field.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o NETCDF_IO/read_attr_from_file.o NETCDF_IO/read_attr_from_file.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o NETCDF_IO/read_mask_from_file.o NETCDF_IO/read_mask_from_file.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o NETCDF_IO/read_var_from_file.o NETCDF_IO/read_var_from_file.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o NETCDF_IO/write_field_to_output.o NETCDF_IO/write_field_to_output.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o NETCDF_IO/write_integral_to_post.o NETCDF_IO/write_integral_to_post.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o NETCDF_IO/write_regions_to_post.o NETCDF_IO/write_regions_to_post.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o NETCDF_IO/write_time_average_to_post.o NETCDF_IO/write_time_average_to_post.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o Functions/Index.o Functions/Index.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o Functions/Index1to4.o Functions/Index1to4.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o Functions/KE_from_vels.o Functions/KE_from_vels.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o Functions/apply_filter_at_point.o Functions/apply_filter_at_point.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o Functions/apply_filter_at_point_for_quadratics.o Functions/apply_filter_at_point_for_quadratics.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o Functions/compute_Pi.o Functions/compute_Pi.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o Functions/compute_Pi_Helmholtz.o Functions/compute_Pi_Helmholtz.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o Functions/compute_Pi_shift_deriv.o Functions/compute_Pi_shift_deriv.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o Functions/compute_Z.o Functions/compute_Z.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o Functions/compute_areas.o Functions/compute_areas.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
mpic++   -c -DDEBUG=0 -w -std=c++14 -O3    -o Functions/compute_div_transport.o Functions/compute_div_transport.cpp -lnetcdf -lhdf5_hl -lhdf5 -lz -lcurl -fopenmp 
In file included from Functions/compute_div_transport.cpp:1:
Functions/../constants.hpp:346:26: error: 'string' is not a member of 'std'
  346 |     const std::map< std::string, std::string > variable_descriptions = {
      |                          ^~~~~~
In file included from Functions/compute_div_transport.cpp:1:
Functions/../constants.hpp:5:1: note: 'std::string' is defined in header '<string>'; did you forget to '#include <string>'?
    4 | #include <map>
  +++ |+#include <string>
    5 | 
In file included from Functions/compute_div_transport.cpp:1:
Functions/../constants.hpp:346:26: error: 'string' is not a member of 'std'
  346 |     const std::map< std::string, std::string > variable_descriptions = {
      |                          ^~~~~~
Functions/../constants.hpp:346:26: note: 'std::string' is defined in header '<string>'; did you forget to '#include <string>'?
Functions/../constants.hpp:346:39: error: 'string' is not a member of 'std'
  346 |     const std::map< std::string, std::string > variable_descriptions = {
      |                                       ^~~~~~
Functions/../constants.hpp:346:39: note: 'std::string' is defined in header '<string>'; did you forget to '#include <string>'?
Functions/../constants.hpp:346:39: error: 'string' is not a member of 'std'
Functions/../constants.hpp:346:39: note: 'std::string' is defined in header '<string>'; did you forget to '#include <string>'?
Functions/../constants.hpp:346:46: error: template argument 1 is invalid
  346 |     const std::map< std::string, std::string > variable_descriptions = {
      |                                              ^
Functions/../constants.hpp:346:46: error: template argument 2 is invalid
Functions/../constants.hpp:346:46: error: template argument 3 is invalid
Functions/../constants.hpp:346:46: error: template argument 4 is invalid
Functions/../constants.hpp:346:48: error: scalar object 'constants::variable_descriptions' requires one element in initializer
  346 |     const std::map< std::string, std::string > variable_descriptions = {
      |                                                ^~~~~~~~~~~~~~~~~~~~~
Functions/../constants.hpp:367:26: error: 'string' is not a member of 'std'
  367 |     const std::map< std::string, std::string > variable_units = {
      |                          ^~~~~~
Functions/../constants.hpp:367:26: note: 'std::string' is defined in header '<string>'; did you forget to '#include <string>'?
Functions/../constants.hpp:367:26: error: 'string' is not a member of 'std'
Functions/../constants.hpp:367:26: note: 'std::string' is defined in header '<string>'; did you forget to '#include <string>'?
Functions/../constants.hpp:367:39: error: 'string' is not a member of 'std'
  367 |     const std::map< std::string, std::string > variable_units = {
      |                                       ^~~~~~
Functions/../constants.hpp:367:39: note: 'std::string' is defined in header '<string>'; did you forget to '#include <string>'?
Functions/../constants.hpp:367:39: error: 'string' is not a member of 'std'
Functions/../constants.hpp:367:39: note: 'std::string' is defined in header '<string>'; did you forget to '#include <string>'?
Functions/../constants.hpp:367:46: error: template argument 1 is invalid
  367 |     const std::map< std::string, std::string > variable_units = {
      |                                              ^
Functions/../constants.hpp:367:46: error: template argument 2 is invalid
Functions/../constants.hpp:367:46: error: template argument 3 is invalid
Functions/../constants.hpp:367:46: error: template argument 4 is invalid
Functions/../constants.hpp:367:48: error: scalar object 'constants::variable_units' requires one element in initializer
  367 |     const std::map< std::string, std::string > variable_units = {
      |                                                ^~~~~~~~~~~~~~
Functions/../constants.hpp:389:16: error: 'string' in namespace 'std' does not name a type
  389 |     const std::string spatial_average_description       = "The lat/lon average computed over each defined region (see region dimension).";
      |                ^~~~~~
Functions/../constants.hpp:389:11: note: 'std::string' is defined in header '<string>'; did you forget to '#include <string>'?
  389 |     const std::string spatial_average_description       = "The lat/lon average computed over each defined region (see region dimension).";
      |           ^~~
Functions/../constants.hpp:397:16: error: 'string' in namespace 'std' does not name a type
  397 |     const std::string zonal_average_description         = "The zonal (longitudinal) average computed at each latitude.";
      |                ^~~~~~
Functions/../constants.hpp:397:11: note: 'std::string' is defined in header '<string>'; did you forget to '#include <string>'?
  397 |     const std::string zonal_average_description         = "The zonal (longitudinal) average computed at each latitude.";
      |           ^~~
Functions/../constants.hpp:405:16: error: 'string' in namespace 'std' does not name a type
  405 |     const std::string time_average_description          = "Time average over the entire provided dataset.";
      |                ^~~~~~
Functions/../constants.hpp:405:11: note: 'std::string' is defined in header '<string>'; did you forget to '#include <string>'?
  405 |     const std::string time_average_description          = "Time average over the entire provided dataset.";
      |           ^~~
Functions/../constants.hpp:415:16: error: 'string' in namespace 'std' does not name a type
  415 |     const std::string OkuboWeiss_average_description    = "Variable binned by Okubo-Weiss parameter (i.e. histogram). "
      |                ^~~~~~
Functions/../constants.hpp:415:11: note: 'std::string' is defined in header '<string>'; did you forget to '#include <string>'?
  415 |     const std::string OkuboWeiss_average_description    = "Variable binned by Okubo-Weiss parameter (i.e. histogram). "
      |           ^~~
make: *** [Makefile:78: Functions/compute_div_transport.o] Error 1

[ashwinvis]

But yes, it manages to compile with gcc9 (with the -w flag, of course)

[bastorer]

Ah, yep, looks like one of the main changes going to gcc10 was including <string> (https://gcc.gnu.org/gcc-10/porting_to.html). I'll add that include statement to constants.hpp, and will read through to see if I can find other files that will need it. I don't have a working gcc10 set-up test on unfortunately.

I'm glad it compiled with gcc9! I'm surprised you needed the -w flag. I compiled with -wAll -wError to turn all warnings into errors and didn't have any pop up.

[ashwinvis]

Isn't it -Wall and -Werror?

[bastorer]

Yep! Sorry, typo :-) I have -Wall on by default, but only added -Werror while I was hunting down the GCC warnings. If you remove -w, do you still get warnings?

[ashwinvis]

An update: I tried to build using conda so I get a full environment based on gcc 9 and there are no warnings, even with -Wall. Here is the environment.yaml file

name: flowsieve-build-env
channels:
 - conda-forge

dependencies:
 - python=3.7
 - numpy=1.17
 - matplotlib=3.*
 - netCDF4=*=mpi*
 - scipy=1.*
 - pip
 - hdf5=*=mpi*
 - openmpi
 - openmpi-mpicxx
 - gcc=9*
 - pip:
   - cmocean
   - pyproj

[bastorer]

Excellent! Glad to hear that it compiles without warnings :-)

I've updated the documentation to note that gcc10+ is currently not supported.

[bastorer]

Are there any changes / updates that you'd like to see on this front, or am I okay to close the ticket?

Thanks for all of your feedback!

[ashwinvis]

Upto you.

It has not been fixed for gcc 10, so if I were you I would keep it open.