search

lwang-astro / PeTar

PeTar is a high-performance N-body code for modelling the evolution of star clusters and tidal streams, including the effect of galactic potential, dynamics of binary and hierarchical system, single and binary stellar evolution.
MIT License
71 stars 19 forks source link

Compilation error with gcc 13.2.0 #46

Closed jobovy closed 9 months ago

jobovy commented 10 months ago

I'm getting a compilation error (invalid preprocessing directive #double) because of this line:

https://github.com/lwang-astro/PeTar/blob/ab1eec771872600091c3520a68493038f558eaea/galpy-interface/galpy_interface.h#L1642

``` galpy_interface.h:1642:18: error: invalid preprocessing directive #double 1642 | # double pot_i = evaluatePotentials(rxy, dz, phi, t, npot, pot_args); | ^~~~~~ make[1]: *** [petar.galpy] Error 1 make[1]: Leaving directory `/home/bovy/tmp/PeTar/galpy-interface' ```

I'm using gcc version 13.2.0 (conda-forge gcc 13.2.0-2). Just commenting that line out allows the compilation to proceed fully.

By the way, the reason that in galpy the potential in C does not depend on phi is that it is only ever used in situations where the potential has to be assumed to be axisymmetric (the potential in C is only used to calculate action-angle coordinates for axisymmetric potentials). But for PeTar it might be useful to have a version of the potential evaluation that does depend on phi? In principle that's not a very difficult change in galpy, although it's hard to test in galpy so might be difficult to implement correctly.

I'm trying to compile PeTar to run an N-body simulation of a globular cluster in the Milky Way's tidal field, but even though the code compiles correctly (when fixing this issue), I get Aborted (core dumped) when I run either the test/sample.sh or test/sample_galpy.sh codes. I'm running these examples exactly as in those files (so setting OMP_STACKSIZE). Do you have any idea why this might be or how I could go about debugging this? Maybe I should use an older version of gcc?

lwang-astro commented 9 months ago

Apologies for the delayed response. In addition to the 'ulimit -s unlimited' as mentioned in my previous email, another way to trace the code causing the crash is to switch on debug mode via configure: configure --with-debug=g Then, use gdb to run the petar commander in sample.sh: OMP_NUM_THREADS=8 OMP_STACKSIZE=128M gdb --arg petar -u 1 -b 500 -t 100.0 -o 5.0 input

In gdb, use run and when the code crashes, use where to show the place where the abort occurred.

jobovy commented 9 months ago

Thanks for the pointer! I ran with gdb and got the following:

``` Starting program: /home/bovy/miniconda3/envs/petar/bin/petar -u 1 -b 500 --galpy-set MWPotential2014 --bse-metallicity 0.02 -t 100.0 -o 5.0 input [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". [New Thread 0x2aaab1772700 (LWP 38417)] [New Thread 0x2aaabb2b5700 (LWP 38418)] [New Thread 0x2aaac4df8700 (LWP 38419)] [New Thread 0x2aaace93b700 (LWP 38420)] [New Thread 0x2aaad847e700 (LWP 38421)] [New Thread 0x2aaae1fc1700 (LWP 38422)] [New Thread 0x2aaaebb04700 (LWP 38423)] //==================================\\ || || || ::::::: ::::::. ::::::. .::::::. || || :: :: : :: : :: || || :::::: :: : ::::::' `:::::. || || :: ::::::' :: `......' || || Framework for Developing || || Particle Simulator || || Version 7.0 (2021/08) || \\==================================// Home : https://github.com/fdps/fdps E-mail : fdps-support@mail.jmlab.jp Licence: MIT (see, https://github.com/FDPS/FDPS/blob/master/LICENSE) Note : Please cite the following papers. - Iwasawa et al. (2016, Publications of the Astronomical Society of Japan, 68, 54) - Namekata et al. (2018, Publications of the Astronomical Society of Japan, 70, 70) Copyright (C) 2015 Masaki Iwasawa, Ataru Tanikawa, Natsuki Hosono, Keigo Nitadori, Takayuki Muranushi, Daisuke Namekata, Kentaro Nomura, Junichiro Makino and many others ******** FDPS has successfully begun. ******** ****************************************** ██████╗ ███████╗████████╗ █████╗ ██████╗ ██╔══██╗██╔════╝╚══██╔══╝██╔══██╗██╔══██╗ ██████╔╝█████╗ ██║ ███████║██████╔╝ ██╔═══╝ ██╔══╝ ██║ ██╔══██║██╔══██╗ ██║ ███████╗ ██║ ██║ ██║██║ ██║ ╚═╝ ╚══════╝ ╚═╝ ╚═╝ ╚═╝╚═╝ ╚═╝ ****************************************** Version: 1211_293 ============== PeTar ================ Online document: (preparing) GitHub page: https://github.com/lwang-astro/PeTar License: MIT Please cite the following papers when you use PeTar for any publications FDPS: see above FPDS Logo message SDAR: Wang L., Nitadori K., Makino J., 2020, MNRAS, 493, 3398 PeTar: Wang L., Iwasawa M., Nitadori K., Makino J., 2020, MNRAS, 497, 536 Galpy: Bovy J., 2015, ApJS, 216, 29 Copyright (C) 2017 Long Wang, Masaki Iwasawa, Keigo Nitadori, Junichiro Makino and many others ===================================== Use quadrupole moment in tree force calculation Use 3rd order tidal tensor method Use Pseudoparticle multipole method Use external potential: Galpy Use orbit-dependent neighbor criterion Check energy of short-range (hard) integration Count isolated particles in hard clusters Use SIMD Calculate profile Use AR TTL method Use slowdown Tree method Use slowdown timescale criterion Print adjust group information Debug mode: PETAR Debug mode: STABLE_CHECK Debug mode: CLUSTER Debug mode: ARTIFICIAL_PARTICLE Debug mode: HARD Debug mode: NAN_CHECK Debug mode: AR ===================================== Input data unit, 0: unknown, referring to G; 1: mass:Msun, length:pc, time:Myr, velocity:pc/Myr: 1 Number of primordial binaries for initialization (assuming the binaries ID=1,2*n_bin): 500 Finishing time of simulation: 100 Output time interval of particle dataset snapshot: 5 Reading data file name: input ----- Finish reading input options of PeTar ----- Add a pre-defined potential set to the potential list, options are: MWPotential2014: MWPotential2014 ----- Finish reading input options of Galpy ----- ----- Parallelization information ----- MPI processors: 1 OMP threads: 8 [New Thread 0x2aab01c60700 (LWP 38425)] [New Thread 0x2aab09c61700 (LWP 38426)] [New Thread 0x2aab11c62700 (LWP 38427)] [New Thread 0x2aab19c63700 (LWP 38428)] [New Thread 0x2aab21c64700 (LWP 38429)] [New Thread 0x2aab29c65700 (LWP 38430)] [New Thread 0x2aab31c66700 (LWP 38431)] Error: Ptcl data reading fails! requiring data number is 4, only obtain 2. Check your input data, whether the consistent features (interrupt mode and external mode) are used in configuring petar and the data generation Thread 1 "petar" received signal SIGABRT, Aborted. 0x00002aaaad767387 in raise () from /lib64/libc.so.6 (gdb) where #0 0x00002aaaad767387 in raise () from /lib64/libc.so.6 #1 0x00002aaaad768a78 in abort () from /lib64/libc.so.6 #2 0x000055555556bfee in Ptcl::readAscii (this=0x2aaaf9a641b0, _fin=0x5555587f6bc0) at ./src/ptcl.hpp:185 #3 0x000055555556c905 in FPSoft::readAscii (this=0x2aaaf9a641b0, fp=0x5555587f6bc0) at ./src/soft_ptcl.hpp:124 #4 0x00005555555f576d in ParticleSimulator::ParticleSystem::readParticleImpl (this=0x7fffffffaaf0, filename=0x7fffffff9a48 "input", format=0x0, header=0x7fffffffaaa8, pFuncPtcl=(void (FPSoft::*)(FPSoft * const, _IO_FILE *)) 0x55555556c8e2 , pFuncHead=(int (FileHeader::*)(FileHeader * const, _IO_FILE *)) 0x55555558336e , open_format=0x5555556e6ffe "r") at /home/bovy/tmp/FDPS-7.0b/src/particle_system.hpp:279 #5 0x00005555555cb0f2 in ParticleSimulator::ParticleSystem::readParticleAscii (this=0x7fffffffaaf0, filename=0x7fffffff9a48 "input", header=...) at /home/bovy/tmp/FDPS-7.0b/src/particle_system.hpp:360 #6 0x000055555559b805 in PeTar::readDataFromFile (this=0x7fffffff93b0) at ./src/petar.hpp:2755 #7 0x000055555555c259 in main (argc=14, argv=0x7fffffffd5d8) at ./src/main.cc:26 ```

Looks like there's an issue with the input data, but I got that by just running the sample_galpy.sh code exactly?

jobovy commented 9 months ago

Wait, I think one issue might be that I didn't compile with --with-interrupt=bse, but if I try adding that, I get the following compilation error:

``` ar rcs libbse.a bse/comenv.o bse/corerd.o bse/deltat.o bse/dgcore.o bse/evolv1.o bse/evolv2.o bse/gntage.o bse/hrdiag.o bse/instar.o bse/kick.o bse/merge.o bse/mix.o bse/mlwind.o bse/mrenv.o bse/print_const.o bse/ran3.o bse/rl.o bse/star.o bse/trdot.o bse/trflow.o bse/zcnsts.o bse/zfuncs.o /home/bovy/miniconda3/envs/petar/bin/g++ -D BSEBBF -D BSE_BASE -g -O0 -fbounds-check -Wall -std=c++17 -fopenmp bse_test.cxx -o petar.bse -L./ -lbse -lgfortran /home/bovy/miniconda3/envs/petar/bin/../lib/gcc/x86_64-conda-linux-gnu/13.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: .//libbse.a(comenv.o): relocation R_X86_64_32S against symbol `value2_' can not be used when making a PIE object; recompile with -fPIE /home/bovy/miniconda3/envs/petar/bin/../lib/gcc/x86_64-conda-linux-gnu/13.2.0/../../../../x86_64-conda-linux-gnu/bin/ld: failed to set dynamic section sizes: bad value collect2: error: ld returned 1 exit status make[1]: *** [petar.bse] Error 1 make[1]: Leaving directory `/home/bovy/tmp/PeTar/bse-interface' make: *** [bse-interface/libbse.a] Error 2 ```

For what I want to do, I don't really need stellar evolution, is there a way to turn it off entirely in the sample_galpy.sh script? I tried removing various options, but keep getting the Ptcl data reading fails error.

lwang-astro commented 9 months ago

Without bse, the input data has different number of columns. You can remove '-s bse' option for petar.init commander. Then, it will generate the input file with correct number of columns for petar:

petar.init  -v kms2pcmyr -f input test.dat.10
OMP_NUM_THREADS=8 OMP_STACKSIZE=128M petar -u 1 -b 500 -t 100.0 -o 5.0 input &>output
jobovy commented 9 months ago

Thanks! I managed to get the code to compile and run I think! First, I fixed the error in my previous comment by adding -no-pie to my CXXFLAGS. Then I was getting weird FORTRAN compilation errors that I correctly assumed were caused by my gfortran version being very old, so I upgraded that (I ended up using GCC/G++/gfortran version 11.4.0 installed using conda). Then I got weird linker errors when linking the FORTRAN code to C++, so I fixed that by adding -lgfortran to my CXXFLAGS (using LIBS or LDFLAGS didn't work). And then things compiled!

I managed to run the sample_galpy.sh code successfully. However, it looks like it doesn't conserve energy very well, is that expected? I assume that's because the time step is too large (I don't think it's explicitly set in the sample_galpy.sh example and I don't think the actual tree time step is printed in the output? The only thing I can find is Tree step number: 2048, which I think means that there are 2048 basic time steps, so then that would be 1/512 Myr; I tried running with a smaller timestep, but energy is still not conserved and the amount of energy drift seems to be about the same regardless of the step size).

Another question that wasn't entirely clear from the README or help functions: are the -t and -o times in Myr (when Myr is the time unit)? And the statement that this has to be 0.5^[integer power] really means that it has to be some integer times 0.5^[integer power] such that it corresponds to an integer number of basic tree time steps?

lwang-astro commented 9 months ago

I think this is expected. When stellar evolution is activated, energy conservation is not very well. Furthermore, the sample_galpy.sh runs a model with 100% binary fraction. The SDAR algorithm may not conserve the energy of perturbed binaries or multiple systems, but the orbital result is reasonable. You can run a model without primordial binaries and without stellar evolution, the energy conservation should be better .

jobovy commented 9 months ago

Okay, indeed, I ran a similar simulation with no stellar evolution and no binaries and energy is indeed conserved at the 1e-7 level, so it looks like all is well. Thanks!

I also checked that the latest master branch compiles under gcc 13.2.0, so I'll close this issue now.