search

davidrosenberg / mlcourse

Machine learning course materials.
https://davidrosenberg.github.io/ml2018
570 stars 267 forks source link

Possible concept check for SGD #52

Open davidrosenberg opened 6 years ago

davidrosenberg commented 6 years ago

For a differentiable function that is Lipschitz continuous with constant L, give a bound on the derivative (for functions mapping R to R). (Write down the limit form of the derivative -- each quotient is bounded by Lipschitz constant, so limit is too. modulo some absolute values ) For functions mapping R^d --> R^d, give a bound on the determinant of the Jacobian. (it's L^d) (use the same strategy but with directional derivatives in coordinate directions -- also https://math.stackexchange.com/questions/1195715/jacobian-determinant-of-lipschitz-function

brett1479 commented 6 years ago

This question (for the gradient) already exists. Do a search for Lipschitz in https://github.com/davidrosenberg/mlcourse/blob/gh-pages/ConceptChecks/AllChecks_sol.pdf . It is in the L1/L2 regularization section.