Prefer to use particles in TracerParticleTypes as tracers for unresolved subhalos

[jchelly]

This PR will modify the code to prefer to use particles of the types specified in TracerParticleTypes as tracers for unresolved ("orphan") subhalos. This should minimize the chance of having orphan subhalos where the tracer particle ceases to exist and we don't have a position for the subhalo.

To do:

• [x] Ensure that TracerIndex is not invalidated by KickNullParticles (#8)
• [x] Add a check that the particle specified by TracerIndex has the expected ID (i.e. index has not been invalidated)
• [x] Stop writing uninitialized positions for orphan subhalos with a non-existent tracer (#11)
• [x] Modify unbinding code to use TracerIndex to pick the tracer particle where possible
• [x] Verify that all orphan subhalos in the COLIBRE test run have an existing tracer particle if we set MinNumTracerPartOfSub>0

[jchelly]

If I set MinNumTracerPartOfSub=1 then in the small COLIBRE test all of the resolved subhalos contain tracer particles, as expected. But there is a small number of orphans (2 of ~40,000) assigned to non-tracer particles. I'm not sure where those are coming from.

[robjmcgibbon]

Hi John, is this ready for review, or are you looking into the orphans assigned to non-tracer particles?

[jchelly]

I'm still looking into the orphans on non-tracers. I've also found that if I update this from master it crashes on a subhalo with zero particles which has never been assigned a most bound particle ID.

[jchelly]

Number of tracers of each type on FLAMINGO L1000N0900/HYDRO_FIDUCIAL with TracerParticleTypes 1 4:

Type 0 : found 1077 most bound particles
Type 1 : found 3484740 most bound particles
Type 2 : found 0 most bound particles
Type 3 : found 0 most bound particles
Type 4 : found 937871 most bound particles
Type 5 : found 2 most bound particles

So mostly correct but there are a few orphans placed on gas and BH particles here too.

[jchelly]

The orphans on the wrong type particles can happen in two ways. One way is much more likely:

• An orphan is created with the wrong tracer type (happens to ~1000 subhalos per snapshot in L1000N0900/HYDRO_FIDUCIAL)
• An orphan changes to a different tracer (happens ~3 times over all snapshots in L1000N0900/HYDRO_FIDUCIAL)

[VictorForouhar]

What else needs to be done before merging?

[jchelly]

Pull requests #20 and #23 are based on this branch, so I guess we need to decide what order we merge things in.

[VictorForouhar]

I will include the changes made by #20 in #23, since I want to have orphans with collisionless tracers only. Hence, #20 should be merged first .

[VictorForouhar]

Note that I have a set of COLIBRE outputs based on the different forks associated to this branch, to check if something has gone wrong at a given point. Masking fix are run with #20, Orphan tracers were run with this branch commit ef54c1f24474c13942ab69559ac001eac21acc64, Multiple tracers on the current version of this branch, and Zero orphans with #23.

[VictorForouhar]

Should we begin merging this branch into master? I can work on that today, and test whether the merged version yields the same objects as this branch, since your other changes are in principle orthogonal to these.

[jchelly]

Yes, I think that makes sense. All of our recent testing has been done on top of this branch anyway.

[VictorForouhar]

I am running the COLIBRE small test box.

[VictorForouhar]

The number of structures is the exact same after merging the branches. The bound mass functions are also the same: Given that merged changes affected very different parts of the code, I believe this small test shows nothing is expected to change dramatically if we were to test on FLAMINGO explicitly.

[VictorForouhar]

Everything looks okay to me; the remaining comments I have do not need to be addressed in this pull request necessarily.

[VictorForouhar]

My latest commit has been tested by seeing if the number of orphans/subhalos change, as well as the bound mass function. I also added a script to check consistency of the tracer IDs between two catalogues. Here is an example comparison between HBT runs done before and after 24f85a1

Testing orphan MostBoundParticleId consistency across two HBT+ catalogues
Opening HBT catalogue: /cosma7/data/dp004/dc-foro1/colibre/hbt_testing/merged_master/006/SubSnap_006.{file_nr}.hdf5 --- DONE
Opening HBT catalogue: /cosma7/data/dp004/dc-foro1/colibre/hbt_testing/no_tracer_saving/006/SubSnap_006.{file_nr}.hdf5 --- DONE
Reference catalogue has 2904 orphans; other catalogue has 2904.
A total of 0 orphans disagree about their MostBoundId.

Testing orphan MostBoundParticleId consistency across two HBT+ catalogues
Opening HBT catalogue: /cosma7/data/dp004/dc-foro1/colibre/hbt_testing/merged_master/007/SubSnap_007.{file_nr}.hdf5 --- DONE
Opening HBT catalogue: /cosma7/data/dp004/dc-foro1/colibre/hbt_testing/no_tracer_saving/007/SubSnap_007.{file_nr}.hdf5 --- DONE
Reference catalogue has 6400 orphans; other catalogue has 6400.
A total of 0 orphans disagree about their MostBoundId.

Testing orphan MostBoundParticleId consistency across two HBT+ catalogues
Opening HBT catalogue: /cosma7/data/dp004/dc-foro1/colibre/hbt_testing/merged_master/008/SubSnap_008.{file_nr}.hdf5 --- DONE
Opening HBT catalogue: /cosma7/data/dp004/dc-foro1/colibre/hbt_testing/no_tracer_saving/008/SubSnap_008.{file_nr}.hdf5 --- DONE
Reference catalogue has 11177 orphans; other catalogue has 11177.
A total of 0 orphans disagree about their MostBoundId.

Testing orphan MostBoundParticleId consistency across two HBT+ catalogues
Opening HBT catalogue: /cosma7/data/dp004/dc-foro1/colibre/hbt_testing/merged_master/009/SubSnap_009.{file_nr}.hdf5 --- DONE
Opening HBT catalogue: /cosma7/data/dp004/dc-foro1/colibre/hbt_testing/no_tracer_saving/009/SubSnap_009.{file_nr}.hdf5 --- DONE
Reference catalogue has 16607 orphans; other catalogue has 16607.
A total of 0 orphans disagree about their MostBoundId.

Testing orphan MostBoundParticleId consistency across two HBT+ catalogues
Opening HBT catalogue: /cosma7/data/dp004/dc-foro1/colibre/hbt_testing/merged_master/010/SubSnap_010.{file_nr}.hdf5 --- DONE
Opening HBT catalogue: /cosma7/data/dp004/dc-foro1/colibre/hbt_testing/no_tracer_saving/010/SubSnap_010.{file_nr}.hdf5 --- DONE
Reference catalogue has 22032 orphans; other catalogue has 22032.
A total of 0 orphans disagree about their MostBoundId.

[VictorForouhar]

Latest change has also been checked against previous version before committing. I do not envision other changes being made on my side, unless @robjmcgibbon spots something I have not.