search

jharno / cubep3m

cosmological n-body code
https://wiki.cita.utoronto.ca/index.php/CubePM
35 stars 11 forks source link

Add a Makefile that uses gfortran #13

Closed martin-g closed 1 year ago

martin-g commented 1 year ago

Uses the Makefile from #14 without -fallow-argument-mismatch. For some reason both gfortran 9.x and 11.x on x86_64 and aarch64 do not support it at Github runners and ubuntu:20.04 Docker image. On my local dev machine (Ubuntu 22.04 x86_64) it is needed.

Add Github Actions CI that builds cubep3m with the new Makefile. Add .gitignore file.

martin-g commented 1 year ago

Related to https://github.com/jharno/cubep3m/issues/12

martin-g commented 1 year ago

Using this new Makefile compiles well but fails at linkage time with:

...
mpif90 -cpp -ffree-line-length-none -g -O3 -fpic -DDIAG -DBINARY -DNGP -DPPINT -DMPI_TIME  -DLRCKCORR -DNGPH -DDISP_MESH  -c update_position.f90
mpif90 -cpp -ffree-line-length-none -g -O3 -fpic -DDIAG -DBINARY -DNGP -DPPINT -DMPI_TIME  -DLRCKCORR -DNGPH -DDISP_MESH  -c variable_initialization.f90
mpif90 -cpp -ffree-line-length-none -g -O3 -fpic -DDIAG -DBINARY -DNGP -DPPINT -DMPI_TIME  -DLRCKCORR -DNGPH -DDISP_MESH  checkpoint.o coarse_cic_mass.o coarse_cic_mass_buffer.o coarse_force.o coarse_force_buffer.o coarse_mass.o coarse_max_dt.o coarse_mesh.o coarse_power.o coarse_velocity.o cubepm.o delete_particles.o fftw3ds.o fine_cic_mass.o fine_cic_mass_buffer.o fine_mesh.o fine_ngp_mass.o fine_velocity.o halofind.o fftw2.o init_projection.o kernel_initialization.o link_list.o move_grid_back.o mpi_initialization.o particle_initialization.o particle_mesh_threaded.o particle_pass.o projection.o report_pair.o report_force.o set_pair.o indexedsort.o timers.o timestep.o update_position.o variable_initialization.o -o cubep3m  -lfftw3_mpi -lsfftw -lm -ldl
/bin/ld: fftw3ds.o: in function `cubepm_fftw_':
/home/martin/git/cubep3m/source_threads/fftw3ds.f90:160: undefined reference to `rfftwnd_f77_mpi_'
/bin/ld: /home/martin/git/cubep3m/source_threads/fftw3ds.f90:158: undefined reference to `rfftwnd_f77_mpi_'
/bin/ld: /home/martin/git/cubep3m/source_threads/fftw3ds.f90:179: undefined reference to `rfftwnd_f77_mpi_destroy_plan_'
/bin/ld: /home/martin/git/cubep3m/source_threads/fftw3ds.f90:127: undefined reference to `rfftw3d_f77_mpi_create_plan_'
/bin/ld: /home/martin/git/cubep3m/source_threads/fftw3ds.f90:129: undefined reference to `rfftw3d_f77_mpi_create_plan_'
/bin/ld: /home/martin/git/cubep3m/source_threads/fftw3ds.f90:180: undefined reference to `rfftwnd_f77_mpi_destroy_plan_'
/bin/ld: fftw2.o: in function `cubepm_fftw2_':
/home/martin/git/cubep3m/source_threads/fftw2.f90:11: undefined reference to `rfftw3d_f77_create_plan_'
/bin/ld: /home/martin/git/cubep3m/source_threads/fftw2.f90:14: undefined reference to `rfftw3d_f77_create_plan_'
/bin/ld: /home/martin/git/cubep3m/source_threads/fftw2.f90:24: undefined reference to `rfftwnd_f77_destroy_plan_'
/bin/ld: /home/martin/git/cubep3m/source_threads/fftw2.f90:21: undefined reference to `rfftwnd_f77_one_complex_to_real_'
/bin/ld: /home/martin/git/cubep3m/source_threads/fftw2.f90:19: undefined reference to `rfftwnd_f77_one_real_to_complex_'
/bin/ld: /home/martin/git/cubep3m/source_threads/fftw2.f90:25: undefined reference to `rfftwnd_f77_destroy_plan_'
/bin/ld: particle_mesh_threaded.o: in function `particle_mesh_':
/home/martin/git/cubep3m/source_threads/particle_mesh_threaded.f90:78: undefined reference to `omp_get_thread_num_'
/bin/ld: timers.o: in function `datestamp_':
/home/martin/git/cubep3m/source_threads/timers.f90:62: undefined reference to `time_'
collect2: error: ld returned 1 exit status
make: *** [Makefile_gfortran_fftw3:18: cubep3m] Error 1

Looks related to http://www.fftw.org/fftw3_doc/Upgrading-from-FFTW-version-2.html#Upgrading-from-FFTW-version-2

Any hints ?

jharno commented 1 year ago

It looks like the FFTW2 is not linking properly, are you sure you use FFTW version 2, compiled with single precision?

martin-g commented 1 year ago

I am sure I use FFTW3 :-) FFTW2 is no more available in Ubuntu repositories and http://www.fftw.org/index.html says:

If you are still using FFTW 2.x, please note that FFTW 2.x was last updated in 1999 and it is obsolete. 
Please upgrade to FFTW 3.x. The API of FFTW 3.x is incompatible with that of FFTW 2.x, for reasons 
of performance and generality (see the [FAQ](http://www.fftw.org/faq/section3.html#fftw2to3) or 
the [manual](http://www.fftw.org/fftw3_doc/Upgrading-from-FFTW-version-2.html)).

I am looking how to upgrade to FFTW3. If I don't succeed then I will use FFTW2.

martin-g commented 1 year ago

I've updated the new Makefile to be based on Makefile_fftw3 and now the error is:

...
mpif90 -cpp -ffree-line-length-none -fallow-argument-mismatch -g -O3 -fpic -rdynamic -mcmodel=medium -DDIAG -DBINARY -DNGP -DPPINT -DMPI_TIME  -DLRCKCORR -DNGPH -DDISP_MESH  -I/usr/include -c fft_coarse.f90
fft_coarse.f90:121:28:

  121 |     type(C_PTR),save :: plan, iplan
      |                            1
Error: Symbol ‘plan’ at (1) already has basic type of INTEGER
fft_coarse.f90:139:80:

  139 |       plan =  fftwf_mpi_plan_dft_r2c_3d(nc_dim, nc_dim, nc_dim, slab, slab_cmplx, mpi_comm_world, FFTW_ESTIMATE)
      |                                                                                1
Error: Symbol ‘slab_cmplx’ at (1) has no IMPLICIT type; did you mean ‘slab_coord’?
make: *** [Makefile_gfortran_fftw3:95: fft_coarse.o] Error 1
martin-g commented 1 year ago

The problem seems to come from https://github.com/jharno/cubep3m/blob/af2fd995e1e5e1a8730aded80b6413dc532847cd/source_threads/fft_coarse.f90#L113 (a symbolic link to cubep3m.fh) which also defines plan: https://github.com/jharno/cubep3m/blob/af2fd995e1e5e1a8730aded80b6413dc532847cd/source_threads/cubep3m.fh#L12

martin-g commented 1 year ago

I have fixed (?!) the last issue by replacing include 'cubepm.fh' with include 'cubepm_fftw3.fh'.

Now the new error is:

mpif90 -cpp -ffree-line-length-none -fallow-argument-mismatch -g -O3 -fpic -rdynamic -mcmodel=medium -DDIAG -DBINARY -DNGP -DPPINT -DMPI_TIME  -DLRCKCORR -DNGPH -DDISP_MESH  -I/usr/include -c fft_coarse.f90
fft_coarse.f90:139:14:

  139 |       plan =  fftwf_mpi_plan_dft_r2c_3d(nc_dim, nc_dim, nc_dim, slab, slab_cmplx, mpi_comm_world, FFTW_ESTIMATE)
      |              1
Error: Type mismatch in argument ‘n0’ at (1); passed INTEGER(4) to INTEGER(8)
fft_coarse.f90:140:15:

  140 |       iplan =  fftwf_mpi_plan_dft_c2r_3d( nc_dim, nc_dim, nc_dim, slab_cmplx, slab, mpi_comm_world, FFTW_ESTIMATE)
      |               1
Error: Type mismatch in argument ‘n0’ at (1); passed INTEGER(4) to INTEGER(8)
make: *** [Makefile_gfortran_fftw3:95: fft_coarse.o] Error 1

The same error happens with include 'cubepm_fftw3_nested.fh' too.

martin-g commented 1 year ago

@jharno The PR is ready for review! CI results for my fork build could be seen here: CI on Linux

martin-g commented 1 year ago

@wuseyu Thank you for your help!