ComputationalRadiationPhysics / picongpu

Performance-Portable Particle-in-Cell Simulations for the Exascale Era :sparkles:
https://picongpu.readthedocs.io
Other
704 stars 217 forks source link

Stationary field at Huygens surface after Laser initialisation with incidentField #4198

Open fabidie opened 2 years ago

fabidie commented 2 years ago

After letting a Laser pulse propagate through the viewing window and it getting reflected at the y_max plane, there is a field visible at the y_min Huygens surface. I assume it is stationary but nearly zero compared to the field of the Laser itself, that is why it is just visible after the pulse has left the window. e_png_yz_0 5_000320 (Laser pulse before leaving the window) e_png_yz_0 5_000400 (Laser pulse after leaving the window/getting reflected, YZ-plane) e_png_yx_0 5_000400 (Laser pulse after leaving the window/getting reflected, YX-plane) I still have to check my assumption with openPMD.

PIConGPU version: 52f5265e43d34679a23c5105c3992d9f17862b50 cluster: hemera-k80

sbastrakov commented 2 years ago

Could you attach the setup? I can have a look tomorrow when I'm back to office.

Also, beware that by default pngs use auto-normalization (independently for each frame). So one would always see some signal there as long as there are non-zero values. There is an option to normalize on a given constant value or for incident field amplitude (except for the Free<> profile), please see png.param

fabidie commented 2 years ago

Checking with openPMD confirms the existence of the stationary field at y_min, and its max intensity is a 0.00065th part of the maximum Laser intensity. statField (left: whole viewing window with propagating pulse; right: same window but zoomed into the y_min area)

fabidie commented 2 years ago

static_field.zip The setup

sbastrakov commented 2 years ago

Oh that relatively high value is unexpected and we should look into it.

Could you share your setup, or is it the IncidentField example as in the repo?

fabidie commented 2 years ago

I used the setup that I put into "static_field.zip" above, using my own parameters which I have written down there in incidentField.param

sbastrakov commented 2 years ago

Oh sorry, I was browsing from phone yesterday and missed that message. Thanks, I am looking into it now.

sbastrakov commented 2 years ago

I was initially even more confused by the gigantic reflection from the right side that your observed rather than the static field. These two things are almost surely unrelated in principle (as the static field occurs before the reflected field arrives), but this very large reflection is already a sign that something is going wrong or unexpected in this simulation.

As with default settings that you used, there should be absorbing boundaries and almost no field reflected back - that reflection should be on order of 1e-4 to 1e-5 of the original field by amplitude, so 1e-8 to 1e-10 in energy, and this is what we observe on some other setups. This is not what happens here at all. I tried both PML and exponential absorber - run with PML (default one) have a very poor absorption of only about 1e-1 in energy, and with exponential dampling it's even worse. So it does not seem like a general absorber bug. I also tested same laser parameters in laser.param and it has same strange behavior, so I also somewhat excluded a bug in incident field implementation or the laser profile used.

After investigating, I think there are two reasons for this behavior (again, unrelated to the static field at the Huygens surface):

As a short summary, i think the setup is currently not well suited for PIConGPU and would be better changed in scale, and to a larger pulse length relative to the time step. And then we should look at potential static fields or other artifacts. Generally, from the literature I saw an estimate of 0.001 amplitude leakage due to the incident field mechanism for the most simplistic schemas. This is about the range you reported, however our implementation should in principle be better as it (supposedly) properly matches numerical dispersion. When we looked at this, it is also important that we choose a numerical dispersion free setup as possible, since dispersion is the main reason for leakage even if we try to match it. Your current parameters are quite off that, but it should be a quick fix with a different field solver or grid parameters. PIConGPU now outputs phase velocity at start of simulation to help controlling it.

sbastrakov commented 2 years ago

@steindev in your experience, what is normal values of PULSE_LENGTH_SI in terms of time step value?

steindev commented 2 years ago

Wow, thanks for checking the numbers. I can believe that the short pulse duration is a problem, since the spectral bandwidth of the pulse is large and the highest frequencies may not be properly resolved by the grid. This results in large dispersion and subsequent pulse lengthening. However, I don't see the connection to the bad absorption yet.

sbastrakov commented 2 years ago

However, I don't see the connection to the bad absorption yet.

I do not have a ready explanation for it, this is just what originally concerned me even more than the stationary field in that simulation as I know from my recent development/experiments with incident fields that reflections were never a problem. To float an idea, our signal (or its finite-difference derivatives that we calculate) may be so poorly represented by the grid, that we cannot properly couple our numerical vacuum to PML and see reflections on this interface. In principle we can check that hypothesis by using a very thick PML with a very gradual growth of $\sigma$. Then we would also see if the reflection happens from the front side (interface of vacuum and PML) or back side (outer boundary of the simulation box) or somehow gradually.

If we are going to investigate this part, i think it also makes sense to look at Fourier image of the original laser.

sbastrakov commented 2 years ago

Also, if we suspect the problem is not with numerical schemes but with floating-point arithmetic (e.g. original choice of scale is somehow bad for PIConGPU as I suspect, see my first bullet point earlier) we could also try rerunning as-is with double precision and see if or how that helps.

Or I guess it's an option to leave it off and switch to a more time-resolved laser and focus on the incident fields.

PrometheusPi commented 2 years ago

@steindev and @fabidie Does your laser definition ensures that the integral of the electric field over the entire volume equal to zero? (Not all our lasers ensure this but with lambda_0 << tau, this is usually negligible.) If it is not ensured, this might explain the static background field (and perhaps the issue with the absorber).