flexcompute / tidy3d

Fast electromagnetic solver (FDTD) at scale.
https://docs.flexcompute.com/projects/tidy3d/en/latest/
GNU Lesser General Public License v2.1
188 stars 44 forks source link

Tidy3d 3.0 restructure 🚀 #1366

Open daquinteroflex opened 10 months ago

daquinteroflex commented 10 months ago

These ideas come from a notion document conversation we have been having, and the idea of this issue is to summarise all the frontend action points or decide what to do further. Just because I was finding it hard to keep track of everything we were talking about.

Low hanging fruit :apple:

Debatable:

Essential changes:

tylerflex commented 10 months ago

I think it would be good to define our 'layers of abstraction' a bit more concretely. Starting from the most fundamental / abstract we might have something like

Core Components Layers

  1. The pydantic classes defining schema through pd.Fields and documentation through docstrings.
  2. Field pydantic.validators (maybe this is combined with 1)
  3. Properties and methods used internally.
  4. Higher level validators (post init validators, pre-upload validators).
  5. Explicitly user-facing methods (eg. Simulation.plot())

(within these core components, there's an import hierarchy we define)

Application Layers (each could be separate packages)

  1. Python client 1a. web API 1b. containers (job, batch)
  2. GUI

It could be good to define what we want to define as "core" components, how we want to separate within those components, and then restructure the package accordingly. This would also help sort out "basic requirements" issue (basic requirements could just be the requirements of the core layers).

tylerflex commented 10 months ago

Also, could be worth thinking about:

  1. A simple, concrete demo (maybe with generic objects) to demonstrate the benefit of potentially separating the schema / pydantic bits from the methods. Could be good to discuss and come up with a few versions of the same demo to compare.
  2. A "test" we can perform on an existing part of the code to tease out where the undesired coupling is. For example, take a file (medium.py) and try splitting every class into its own file. If we can't do this without circular imports or forward reference hacking in pydantic, we have identified some problem. (note, this happened in geometry when we refactored it into separate files, but we kept it).