Particle ID is not always set for particle-probe species

[Status-Mirror]

Particle probes correctly output particle ID - however, the particle ID values are modified after passing a probe. I have included a basic input deck at the bottom of this report which demonstrates the problem. Here we set up a beam of 200000 electrons passing two probes in space. When using the compilation flags:

• -DPROBE_TIME
• -DPARTICLE_ID

the first probe outputs particles ID 1-200000. This is expected, as all particles pass the first probe. All particles then pass the second probe, which records ID values 200001-400000. It seems like the particle ID values for the electrons have been recalculated after passing the first probe.

This issue was first identified by Ethan at the University of York in a private communication to me.

Input deck:

begin:control  
  nx = 200          
  ny = 200            
  t_end = 200.0e-15      
  x_min = 0.0e-6    
  x_max = 4.0e-6      
  y_min = -2.0e-6
  y_max = 2.0e-6  
end:control  

begin:boundaries  
  bc_x_min = simple_outflow  
  bc_x_max = simple_outflow  
  bc_y_min = periodic 
  bc_y_max = periodic
end:boundaries  

begin:species
  name = Electron
  npart = 10 * ny * 100
  density = if (x lt 10*dx, 1e10, 0) 
  drift_x = 2.7e-21 
  identify:electron
end:species   

begin:output
  dt_snapshot = 10.e-15   
  grid = always 
  particle_probes = always  
end:output 

begin:probe
   name = first_probe
   point = (1.0e-6, 0.0)
   normal = (1.0, 0.0)
   ek_min = 0.0
   ek_max = -1.0
   include_species:Electron
   dumpmask = always
end:probe

begin:probe
   name = second_probe
   point = (2.0e-6, 0.0)
   normal = (1.0, 0.0)
   ek_min = 0.0
   ek_max = -1.0
   include_species:Electron
   dumpmask = always
end:probe

[Status-Mirror]

This issue is very similar to issue #431 - we only have a problem when we do not output "id". Currently, ID variables are only assigned to particles when we go to output ID.

In the example deck I provided, we never attempt to output the electron ID, so no electron ID is every assigned. All electron ID values are 0 in this simulation, so when the particle is copied to the probe output list, the ID is still 0. The particle is assigned a false ID before being output, but this is simply 1 plus the last ID.

We can fix this by forcing EPOCH to assign an ID to any species which has an attached particle probe. Our proposed fix performs this in the push_particles routine, before the actual particle push begins. We have chosen this routine, as the code which checks if particles pass the probe is also here.

[DanRRRR]

Related question: if i define DEFINES += $(D)PARTICLE_ID which is by default OFF what specifically happens what may detriment the code run speed, allocated memory etc? Are burdens too high to keep default OFF? If slowdown is less than few % then who cares? And memory becomes cheaper and cheaper very fast. I'd prefer it always to be ON. Same is done in some other codes where IDs for all particles are always ON, if they are not needed today, they might be needed later

[Status-Mirror]

When running EPOCH in parallel, each processor only sees a fraction of the total simulation. As particles move through space, they can move from regions controlled by one processor, to another. When this happens, the particle data needs to be transferred between processors, and the more particle variables to transfer (like particle ID, or optical emission depths for bremsstrahlung or qed emission), the slower the code runs. MPI transfer is a slow process, so we want to mimise the performace hit.

EPOCH is designed to run efficiently on massive supercomputers with huge processor counts. By default, EPOCH only runs the core algorithms with minimal particle data. Even if it's a tiny slowdown, when you run on massive, expensive clusters, the cost-saving is worth it.