search

SciML / ModelingToolkit.jl

An acausal modeling framework for automatically parallelized scientific machine learning (SciML) in Julia. A computer algebra system for integrated symbolics for physics-informed machine learning and automated transformations of differential equations
https://mtk.sciml.ai/dev/
Other
1.41k stars 204 forks source link

Creating an "ImageProblem" problem type in MTK #1012

Closed 00krishna closed 7 months ago

00krishna commented 3 years ago

Many people are working on Image Processing and Image Segmentation problems, using classical, variational, and neural network methods. These types of problems are challenging to program because they include a PDE component, an optimization component, and they can be numerically intensive. Hence, it might be a good idea to create an "ImageSegmentation" or "ImageProblem" problem definition under MTK. The idea is that a user could setup an ImageProblem with MTK, define the solver to use (ADMM, Split-Bregman, etc.), and define any constraints--like the Total Variation Norm. Then MTK could handle the setup and execution of the problem.

Users could also define their own solvers, or customize the problem for different applications. The details would need to be worked out. But starting with some simple examples such as:

  1. Chan-Vese Segmentation
  2. L2 Total variation denoising
  3. Something with a level set
  4. Something with a graph cuts based segmentation

The corresponding algorithms for optimization would be ADMM, Split-Bregman, and others from ProximialAlgorithms.jl.

The developers can let us know if it is better to create a new Problem Type, or include these image problems under an existing problem type.

There is some interest in develop this and some tutorials by @kellertuer and @Dale-Black .

kellertuer commented 3 years ago

Just a small remark, I would call TV (or even TGV or what ever you like) not a constraint but a regulariser? It would just be “expensive” to have a large TV value, but not constraint to zero TV (since that would limit you to constant functions).

Really an equality constraint I would set for example for pure inpainting, where you want the image to be equal to given data at the pixels that are known.

ChrisRackauckas commented 3 years ago

@tjdiamandis might have thoughts.

ChrisRackauckas commented 7 months ago

I'm not sure this makes sense here as a mathematical primitive. Domain-specific extensions live in packages.