search

lasso-net / lassonet

Feature selection in neural networks
MIT License
209 stars 52 forks source link

LassoNet for Multitask Learning #41

Open pranavsankhe opened 1 year ago

pranavsankhe commented 1 year ago

Can lassonet be extended to multitask learning neural networks? If so, how do I go about implementing it? TIA

louisabraham commented 1 year ago

Hi, thank you for this very interesting question!

Multitask learning involves sharing weights from the first layers, then specializing them for different tasks.

In LassoNet, there is a skip connection with a L1 regulatization linking the input to the output. If you want the masks to be the same for different tasks, you could first try a multidimensional regression.

There might be gain to split the architecture at some point, which would involve some smart handling of the dimensions in model.py