There was a bug in source_moab.F90 where memory continually increased as the number of particle histories increased. This mean running multiple jobs crashed tux-c1 (150million NPS ~=24 gb of memory used...)
This was due to arrays for entity and vertex information being allocated new memory each time the code sampled within a voxel, while keeping the old data in memory. I was able to solve this by using 'save' on the relevant variables. (This was a hypothesis, aka lucky guess) Quick testing indicates this does not affect the problem results, and perhaps slightly speeds up the code. Relevant commit: 83a33c264895688685287bab29a0940660a428d8
I implemented this change for the voxel sampling code, and also for voxel volume calculation code, which reduces the memory further.
One potential, minor flaw this may introduce is issues handling mesh with both tet and hex voxels. I think this will be handled fine, but it ought to tested. (i.e. manual creation of a .vtk file with a hex and a tet; This could be a good task for a new guy :+1:)
Finally, this was a useful script for not having to actively watch the memory usage:
There was a bug in source_moab.F90 where memory continually increased as the number of particle histories increased. This mean running multiple jobs crashed tux-c1 (150million NPS ~=24 gb of memory used...)
This was due to arrays for entity and vertex information being allocated new memory each time the code sampled within a voxel, while keeping the old data in memory. I was able to solve this by using 'save' on the relevant variables. (This was a hypothesis, aka lucky guess) Quick testing indicates this does not affect the problem results, and perhaps slightly speeds up the code. Relevant commit: 83a33c264895688685287bab29a0940660a428d8
I implemented this change for the voxel sampling code, and also for voxel volume calculation code, which reduces the memory further.
One potential, minor flaw this may introduce is issues handling mesh with both tet and hex voxels. I think this will be handled fine, but it ought to tested. (i.e. manual creation of a .vtk file with a hex and a tet; This could be a good task for a new guy :+1:)
Finally, this was a useful script for not having to actively watch the memory usage: