search

pynbody / genetIC

The GM initial conditions generator
GNU General Public License v3.0
21 stars 8 forks source link

Assertion error when generating double/triple zooms near grid boundaries #90

Closed Martin-Rey closed 3 years ago

Martin-Rey commented 3 years ago

HI all,

I am hitting an assertion error when generating a zoom on a specific edge case (zooming on other objects in the box proceeds as usual). My setup is quite involved, with three nested zooms as the "volume" and I am trying to generate the fourth, final zoom level on my halo of interest.

genetIC v1.1.1, compiled: Apr 29 2021, 13:50:27
              runtime:  Apr 30 2021, 09:16:24

git HEAD:07dbc11
At the time of compilation, the following files had been modified:
  Makefile

 2021-04-30 09:16:24   
 2021-04-30 09:16:24   Limiting number of FFTW Threads to 8, because FFTW on Mac OS seems to become slow beyond this point.
 2021-04-30 09:16:24   To disable this behaviour, recompile with -DIGNORE_APPLE_FFTW_THREAD_LIMIT
 2021-04-30 09:16:24   OpenMP parts of the code will still run with 12 threads.
 2021-04-30 09:16:24   
 2021-04-30 09:16:24   Using post 2015 CAMB transfer function
 2021-04-30 09:16:24   Initialized the base grid:
 2021-04-30 09:16:24     Box length            = 67.32 Mpc/h
 2021-04-30 09:16:24     n                     = 256
 2021-04-30 09:16:24     dx                    = 0.262969
 2021-04-30 09:16:24   Adding real grid with resolution 256
 2021-04-30 09:16:24   Loading ../void/void.iords.txt
 2021-04-30 09:16:25     -> total number of particle IDs loaded is 119826
 2021-04-30 09:16:25        total number of accessible cells flagged is 119826
 2021-04-30 09:16:25   Centre of region is 17.8711588248 58.6582382854 25.0155118907
 2021-04-30 09:16:25   Initialized a zoom region:
 2021-04-30 09:16:25     Subbox length         = 33.66 Mpc/h
 2021-04-30 09:16:25     n                     = 256
 2021-04-30 09:16:25     dx                    = 0.131484375
 2021-04-30 09:16:25     Zoom factor           = 2
 2021-04-30 09:16:25     Num particles         = 119826
 2021-04-30 09:16:25     Low-left corner in parent grid = (3, 161, 35)
 2021-04-30 09:16:25     Low-left corner (h**-1 Mpc)    = 0.78890625, 42.33796875, 9.20390625
 2021-04-30 09:16:25   Adding real grid with resolution 256
 2021-04-30 09:16:25   Adding real grid with resolution 512
 2021-04-30 09:16:25     Total particles       = 33554432
 2021-04-30 09:16:35   Initialized a zoom region:
 2021-04-30 09:16:35     Subbox length         = 16.83 Mpc/h
 2021-04-30 09:16:35     n                     = 256
 2021-04-30 09:16:35     dx                    = 0.0657421875
 2021-04-30 09:16:35     Zoom factor           = 2
 2021-04-30 09:16:35     Num particles         = 1481544
 2021-04-30 09:16:35     Low-left corner in parent grid = (65, 59, 55)
 2021-04-30 09:16:35     Low-left corner (h**-1 Mpc)    = 9.335390625, 50.095546875, 16.435546875
 2021-04-30 09:16:35   Adding real grid with resolution 256
 2021-04-30 09:16:35   Adding real grid with resolution 512
 2021-04-30 09:16:35   Adding real grid with resolution 1024
 2021-04-30 09:16:35     Total particles       = 50331648
 2021-04-30 09:16:58   Initialized a zoom region:
 2021-04-30 09:16:58     Subbox length         = 8.415 Mpc/h
 2021-04-30 09:16:58     n                     = 512
 2021-04-30 09:16:58     dx                    = 0.016435546875
 2021-04-30 09:16:58     Zoom factor           = 2
 2021-04-30 09:16:58     Num particles         = 1670998
 2021-04-30 09:16:58     Low-left corner in parent grid = (65, 66, 66)
 2021-04-30 09:16:58     Low-left corner (h**-1 Mpc)    = 13.6086328125, 54.43453125, 20.77453125
 2021-04-30 09:16:58   Adding real grid with resolution 256
 2021-04-30 09:16:58   Adding real grid with resolution 512
 2021-04-30 09:16:58   Adding real grid with resolution 1024
 2021-04-30 09:16:58   Adding virtual grid with effective resolution 2048
 2021-04-30 09:16:58   Adding real grid with resolution 4096
 2021-04-30 09:16:58     Total particles       = 201326592
 2021-04-30 09:18:44   Loading ./halo318_5rvir.txt
 2021-04-30 09:18:44     -> total number of particle IDs loaded is 19767
 2021-04-30 09:18:44        total number of accessible cells flagged is 23329
 2021-04-30 09:18:44   Centre of region is 20.5155946842 55.0619393569 25.4002363884
Assertion failed: (this->containsPoint(result)), function getCentroidFromCoordinate, file /Users/martinrey/Documents/genetIC/genetIC/src/simulation/grid/grid.hpp, line 646.

The assertion fail arises from the getCentroidFromCoordinate in this line: https://github.com/pynbody/genetIC/blob/07dbc11d6304b40b646e9ec18c6964a0f494a800/genetIC/src/ic.hpp#L440-L449 when opening the final zoom, but the problem is generated further up here https://github.com/pynbody/genetIC/blob/07dbc11d6304b40b646e9ec18c6964a0f494a800/genetIC/src/ic.hpp#L425-L426

The bug is similar in logic to #76 -- the window symmetric expansion always assume that the zoom window can be wrapped around the base grid, which is true for first-level zooms, but not for double or triple zoom setups. Specifically, in my case, my object's fourth grid is near the edge of the third zoom, and the lower left corner gets wrapped during the symmetric expansion by these lines: https://github.com/pynbody/genetIC/blob/07dbc11d6304b40b646e9ec18c6964a0f494a800/genetIC/src/simulation/window.hpp#L168-L179 leading to assert error later down the line

I am unsure how to solve this, and I am not super familiar with the code introduced in #76. It feels like it requires the window to have two mechanics (i) to expand symmetrically as it currently does and (ii) to expand within the bouding box of the parent grid. But both these modes have to correctly wrap in the overall simulation domain, even if they cannot wrap from the parent grid. Any input @apontzen ?

Cheers, Martin

apontzen commented 3 years ago

I am closing this issue as I believe it was fixed by #91