SciML / SciMLSensitivity.jl

A component of the DiffEq ecosystem for enabling sensitivity analysis for scientific machine learning (SciML). Optimize-then-discretize, discretize-then-optimize, adjoint methods, and more for ODEs, SDEs, DDEs, DAEs, etc.
https://docs.sciml.ai/SciMLSensitivity/stable/
Other
328 stars 71 forks source link

Adjoint method docstrings #1025

Open ArnoStrouwen opened 6 months ago

ArnoStrouwen commented 6 months ago

the explanation of the chunk size keyword does not make sense, it mentions autojacvec=false, but autojacvec=false corresponds to false: the Jacobian is constructed via FiniteDiff.jl. The autodiff keyword seems superfluous given the autojacvec = true option: true: the Jacobian is constructed via ForwardDiff.jl

help?> QuadratureAdjoint()
  QuadratureAdjoint{CS, AD, FDT, VJP} <: AbstractAdjointSensitivityAlgorithm{CS, AD, FDT}

  An implementation of adjoint sensitivity analysis which develops a full continuous solution of the reverse solve in order to perform a post-ODE quadrature. This method requires the dense solution and will ignore saving arguments during the gradient
  calculation. The tolerances in the constructor control the inner quadrature.

  This method is O(n^3 + p) for stiff / implicit equations (as opposed to the O((n+p)^3) scaling of BacksolveAdjoint and InterpolatingAdjoint), and thus is much more compute efficient. However, it requires holding a dense reverse pass and is thus memory
  intensive.

  Constructor
  ===========

  QuadratureAdjoint(; chunk_size = 0, autodiff = true,
      diff_type = Val{:central},
      autojacvec = nothing, abstol = 1e-6,
      reltol = 1e-3)

  Keyword Arguments
  =================

    •  autodiff: Use automatic differentiation for constructing the Jacobian if the Jacobian needs to be constructed. Defaults to true.

    •  chunk_size: Chunk size for forward-mode differentiation if full Jacobians are built (autojacvec=false and autodiff=true). Default is 0 for automatic choice of chunk size.

    •  diff_type: The method used by FiniteDiff.jl for constructing the Jacobian if the full Jacobian is required with autodiff=false.

    •  autojacvec: Calculate the vector-Jacobian product (J'*v) via automatic differentiation with special seeding. The total set of choices are:
       • nothing: uses an automatic algorithm to automatically choose the vjp. This is the default and recommended for most users.
       • false: the Jacobian is constructed via FiniteDiff.jl
       • true: the Jacobian is constructed via ForwardDiff.jl
       • TrackerVJP: Uses Tracker.jl for the vjp.
       • ZygoteVJP: Uses Zygote.jl for the vjp.
       • EnzymeVJP: Uses Enzyme.jl for the vjp.
       • ReverseDiffVJP(compile=false): Uses ReverseDiff.jl for the vjp. compile is a boolean for whether to precompile the tape, which should only be done if there are no branches (if or while statements) in the f function.

    •  abstol: absolute tolerance for the quadrature calculation

    •  reltol: relative tolerance for the quadrature calculation

  For more details on the vjp choices, please consult the sensitivity algorithms documentation page or the docstrings of the vjp types.
ChrisRackauckas commented 6 months ago

The autodiff keyword seems superfluous given the autojacvec = true option: true: the Jacobian is constructed via ForwardDiff.jl

I'm not sure that's correct. I think you need autojacvec false and then it looks at autodiff? I'd check the code and fix the docstring to that.