Adjoint Master Plan

[tylerflex]

October 2024 >

• [ ] short term

  • [x] look into gradients again
  • [ ] publish notebooks (field projection + photonic crystal)
  • [x] Update Readme including how to use jax / pytorch, not saying it's an "experimental feature"
  • [x] yannick merge open PRs? (field projection, local cache, faqs, etc)

• [ ] features

  • [ ] slab bounds (polyslab.slab_bounds, cylinder.length)
  • [ ] triangular mesh -> implement other shapes (eg sphere) using a mesh conversion
  • [x] server side field projection
  • [ ] polyslab dilation / sidewall angle
  • [ ] field time monitor
  • [ ] autograd for resonance finder
  • [ ] numerical dA/dp
  • [ ] multi-frequency fitting.
  • [ ] anisotropic medium (custom and regular)

• [ ] multi-physics

  • [ ] nonlinear adjoint
  • [ ] adjoint mode solver
  • [ ] adjoint heat solver
  • [ ] adjoint component modeler
  • [ ] adjoint photon forge

• [ ] performance

  • [x] make data arrays faster and easier to use
  • [x] speedup geometry group with many structures / vjp generalize
  • [ ] 1 forward N adjoint
  • [ ] simultaneous fwd and adjoint
  • [ ] lossy compression for adjoint field monitors
  • [ ] unstructured point monitors

• [ ] examples

  • [ ] Industry relevant adjoint examples
  • [ ] invdes implementations of existing notebooks
  • [ ] convert adjoint notebooks to autograd
  • [ ] standardize variable names in notebooks
  • [ ] pytorch example
  • [ ] optimization with constraints example

• [ ] invdes improvements

  • [ ] Invdes python web API hookup
  • [ ] Invdes redesign / API
  • [ ] start with given permittivity
  • [ ] parameter scheduling
  • [ ] shape design region
  • [ ] unify with design plugin? or add gradient based methods to design plugin

Older

Roughly ordered by priority within each section

Tyler

• [ ] Small / Polish

  • [ ] Dont error if only one sim in batch doesn't have tracers (no adjoint sources)
  • [x] Make box gradients work in 1D (I think it ignores 1D field data by default), test on slab.
  • [x] Why are polyslab gradients larger magnitude in new code? (notebook 8)
  • [x] Better handling of actual web.run() errors
  • [ ] BatchData from _run_async() - [ ]

• [x] improve test coverage

  • [x] No adjoint sources warning
  • [x] autograd web raise exceptions -> warning
  • [x] invalid inputs for autograd dont run autograd

• [x] Server side autograd gradient processing (with momchil)

• [ ] Multi-frequency.

  • [x] Post-normalization with single broadband adjoint source
  • [x] WDM notebook.
  • [ ] Broadband Multiple Sources (fitting approach)

• [ ] Monitor Support

  • [x] FieldMonitor
    • [x] differentiable DataArray interp
      • [x] Intensity
        • [x] Quickstart.
        • [x] Metasurface notebook.
      • [x] Flux
      • [x] Poynting vector.
  • [ ] FieldTimeMonitor
  • [ ] Server-side FieldProjectionMonitors

• [ ] Medium support

  • [x] Anisotropic custom medium.
  • [ ] AnisotropicMedium
  • [x] Dispersive Mediums
    • [x] Pole Residue (and other types)
    • [x] Custom Pole Residue (and other types)
      • [x] notebook on topology optimization with metals

• [ ] Geometry Support

  • [x] GeometryGroup
    • [x] Metalens Demo
  • [x] ComplexPolyslab
  • [ ] all other (traceable) Polyslab fields
  • [x] Cylinder
    • [ ] PhC demo
  • [ ] Sphere
  • [ ] Trianglemesh

Yannick

• [ ] Notebook demo for autograd utilities
• [x] differentiable DataArray interp
• [ ] Invdes improvements
  • [x] Schema-based post processing
  • [x] GUI support (with Emerson)
  • [x] Web API support
  • [ ] More optimizers. (Ya)
  • [ ] More design regions. (Ya)
• [ ] Wrappers for other auto diff (Ya)
  • [ ] Jax
  • [x] Pytorch
  • [ ] Tensorflow

[nkvij]

Hello, I am a Tidy3D user and am interested in doing inverse design on photonic crystal cavities for which I would require Adjoint support for Cylinder geometry, FieldTimeMonitor and FieldMonitor. Is it possible for you guys to bump up these things in priority? It would also be great if you could give me an approximate timeline for these features. Also, apologies if this is the wrong place for this comment. Thank you, Neelesh

[tylerflex]

Hi @nkvij thanks for your feedback.

• FieldMonitor is supported as of the latest release (2.7.2) and you can differentiate through various operations on the field monitor output, such as flux, poynting and intensity.
• Cylinder is in production, there's actually a PR open for it https://github.com/flexcompute/tidy3d/pull/1791 but it will need some more work. I'd anticipate it will be out within the next 2-3 weeks.
• FieldTimeMonitor is a bit trickier. In principle it shouldn't be too hard, but we need more bandwidth to work on it. So I'll bump the priority (you're the first person who asked) but I wouldn't expect that one for the next 2-3 months.

Thanks for your patience. We're going through a bit of a clean-up period to make the feature more stable and tested after the very fast feature implementation over the past couple months, but once we finish that up, we'll definitely start on cylinder and field time!

[momchil-flex]

Hi @nkvij , do you also need our ResonanceFinder to be differentiable and not just the FieldTimeMonitor? Because that's another thing that's not even on the map yet.

By the way maybe you should consider legume :)

[nkvij]

Hey @momchil-flex, just saw your comment. Yes, I would need Resonancefinder to be differentiable too. I forgot that it was separate from FieldTimeMonitor.

Haha that's a great pitch for Legume. However, it would be nice to be able to inverse design 1D photonic crystals too :)

[nkvij]

Hello, as we are reaching the 2-month mark, I wanted to check in if there are any updates regarding this. Thank you so much!

Hi @nkvij thanks for your feedback.

• FieldMonitor is supported as of the latest release (2.7.2) and you can differentiate through various operations on the field monitor output, such as flux, poynting and intensity.
• Cylinder is in production, there's actually a PR open for it 🗻 autograd Cylinder #1791 but it will need some more work. I'd anticipate it will be out within the next 2-3 weeks.
• FieldTimeMonitor is a bit trickier. In principle it shouldn't be too hard, but we need more bandwidth to work on it. So I'll bump the priority (you're the first person who asked) but I wouldn't expect that one for the next 2-3 months.

Thanks for your patience. We're going through a bit of a clean-up period to make the feature more stable and tested after the very fast feature implementation over the past couple months, but once we finish that up, we'll definitely start on cylinder and field time!

[tylerflex]

Hi @nkvij ,

Cylinder is working, but we have not started work on FieldTimeMonitor or ResonanceFinder. I looked into the field time monitor support and I think it will require substantial changes. Since we have a number of things on our roadmap which are higher urgency (such as some performance improvements), I don't anticipate being able to work on this for at least a few more months.

In the meantime, it might be worth seeing if you can find a workaround solution. For example, perhaps you can run your simulations with FieldTimeMonitor and ResonanceFinder, find the resonance frequency, and then run a differentiable simulation using a FieldMonitor at that frequency?

Given your concern, I'll bump the priority of this feature and will keep you updated once we start work on it.

[nkvij]

How do I use the cylinder geometry?

I will try and figure out a workaround using only the FieldMonitor. Thank you for the update!

[tylerflex]

The cylinder just works when your objective function arguments ultimately give rise to either the center or the radius of a td.Cylinder. For example, an objective function like

def f(x):
    radius = x + 2
    cyl = td.Cylinder(radius=radius, ...)
    # make a sim containing this cylinder
    # figure of merit, etc.

see cell [9] of this unpublished notebook for an example for photonic crystals

https://github.com/flexcompute/tidy3d-notebooks/blob/e6501bf89419fbefcd3566b1eee1dc062dbc5876/Autograd18PhotonicCrystal.ipynb