PumasAI / SimpleChains.jl

Simple chains
MIT License
233 stars 14 forks source link

Add documentation for L1/L2 penalties #151

Open era127 opened 1 year ago

era127 commented 1 year ago

Would it be possible to add documentation to the simple mlp example to show how to add the L1 and L2 penalties. Thanks!

jmsull commented 1 year ago

You can do it like this:

using SimpleChains
sc = SimpleChain(
    static(1),
    TurboDense{true}(tanh, 128), 
    TurboDense{false}(identity, 1), 
    )
p = SimpleChains.init_params(sc);
G = SimpleChains.alloc_threaded_grad(sc); 

Y = # some data...
λ = # some penalty...
sc_reg = FrontLastPenalty(SimpleChains.add_loss(sc, SquaredLoss(Y)),
                                    L2Penalty(λ), L2Penalty(λ) )