FluxML / MLJFlux.jl

Wrapping deep learning models from the package Flux.jl for use in the MLJ.jl toolbox
http://fluxml.ai/MLJFlux.jl/
MIT License
145 stars 17 forks source link

Add L2/L1 regularization #179

Closed ablaom closed 3 years ago

ablaom commented 3 years ago

This PR:

@ToucheSir Further to Julia Discourse discussion, no change was actually necessary to the coretrain! loop to add regularization. It's just that the loss function passed to this loop now depends on the chain (Flux model), when L2/L1 regularisation parameters are non-trivial. To avoid the array mutation error I needed to avoid broadcasting in the computation of the penalty here. Any performance suggestions re these two bits of code appreciated.

codecov-commenter commented 3 years ago

Codecov Report

Merging #179 (67409a5) into dev (a40bdfd) will decrease coverage by 0.14%. The diff coverage is 95.12%.

Impacted file tree graph

@@            Coverage Diff             @@
##              dev     #179      +/-   ##
==========================================
- Coverage   90.74%   90.59%   -0.15%     
==========================================
  Files           8        9       +1     
  Lines         216      234      +18     
==========================================
+ Hits          196      212      +16     
- Misses         20       22       +2     
Impacted Files Coverage Δ
src/types.jl 85.71% <0.00%> (-14.29%) :arrow_down:
src/common.jl 96.77% <100.00%> (+0.22%) :arrow_up:
src/core.jl 90.00% <100.00%> (-0.79%) :arrow_down:
src/penalized_losses.jl 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update a40bdfd...67409a5. Read the comment docs.

ablaom commented 3 years ago

@ayush-1506 Would you like to review this PR? I've tested it locally on a GPU.

@DilumAluthge How do I put the GPU tests back for PR's onto dev?

ayush-1506 commented 3 years ago

@ablaom Sure, please give me a day.