Vanishing gradients

[kdotom]

This is likely user error, but:

My goal is to optimize a photonic waveguide crossing in 3D by using the 04-Splitter.ipynb as a template. The structure optimizes just fine in 2D, but when moving to 3D, for whatever reason, the gradients appear to be vanishing after about iteration 2 of each beta epoch.

The setup has:

• source: Two sources, one for the north input and one for the west input
• sim: Two simulation objects corresponding to the sources
• opt: Two OptimizationProblem objects corresponding to the simulations
• f0: The f0 term taken to be the mean of the individual f0's from the two optimization objects
• dJ_du: Gradients taken to be the mean of the individual dJ_du's from the optimization objects
• No symmetry: Removed symmetry for generality
• The same optimization infrastructure as the notebook (i.e. autograd + nlopt)

So far, I've:

• Modified the original splitter to be a waveguide crossing with a design region.
• Made the sources and monitors 2D planes as opposed to lines.
• Made the waveguides 3D volumes as opposed to flat surfaces
• Kept the design region variables 2D
• Made the design region 3D

What I'm seeing is:

• Iteration 1: Relatively level (0.5) design-region with small traces of a solution.
• Iteration 2: Semi-defined structure which looks like the optimizer is on the right track (and well-defined gradients)
• Iteration 3+: Fading structure definition and fading gradients. It fades more with each iteration.
• This cycle repeats each epoch including the Iteration 1 trend even if the last cycle of the previous epoch was well-defined.

For mapping, I've tried:

• Beta values between 2 and 256. (2, 4, 8, 16, 32, etc.) with eta_i = 0.55
• Eta_i values between 0.51 and 0.9 with beta values and number of epochs similar to one of @smartalecH's papers.

As noted at the beginning, the setup works just fine when optimizing in 2D, but struggles with 3D.

I might be missing something simple, so just wanted to reach out and see if anyone has seen and solved this.

Thanks in advance. -K

[smartalecH]

I don't see anything "obviously wrong" from your description above. If it works fine in 2D, just not in 3D, then it's probably something with the simulation itself (ie not with the optimization, the gradient computation, or even the hyperparameters).

So maybe look a bit more carefully at the FOM itself and the values that are being returned. Do they seem logical? Does their evolution seem logical? You mentioned you are taking a mean... look at all the individual FOMs and try to pick up on any trends. Make sure they are all on the same scale and that there aren't any weird outliers.

I would look at the fields themselves at a few different points in time too. Maybe even swap in a CW source.

It would be worth it to look at all the individual gradients as a sanity check too.

The goal is to figure out where in the pipeline things are breaking down. This is tricky, since the pipeline consists of so many steps that could have bugs. So start with some assumptions to narrow down your search, and work from there.

(@Oskooi we mentioned at one point we should add a page dedicated toward debugging adjoint optimization problems... we should probably revisit that)

[oskooi]

My goal is to optimize a photonic waveguide crossing in 3D by using the 04-Splitter.ipynb

In addition to the notebook examples, it might be useful to review this recently added tutorial which provides helpful tips for using the adjoint-solver module: https://meep.readthedocs.io/en/latest/Python_Tutorials/Adjoint_Solver/#broadband-waveguide-mode-converter-with-minimum-feature-size.

(We are also planning to add 3d tutorial examples. Stay tuned.)

[kdotom]

I put visuals on everything to check placement and looked at values at different points in the pipeline. FOM values were going the other direction, so (after a bit of reading) I rebuilt it ab initio and used an even simpler cost function than I was using. + It's now working! Thanks for the tips @oskooi @smartalecH