search

greenelab / pancancer-evaluation

Evaluating genome-wide prediction of driver mutations using pan-cancer data
BSD 3-Clause "New" or "Revised" License
9 stars 3 forks source link

TCGA -> CCLE neural network experiments #76

Closed jjc2718 closed 1 year ago

jjc2718 commented 1 year ago

To follow up on our LASSO experiments, we wanted to try training neural networks, and see if generalization trends looked like the ones we saw when varying the LASSO parameter. We started by using training epochs in a similar way, with models trained for fewer epochs being considered more "regularized" and models trained for longer less so.

We didn't really see too much of the regularization effect we were looking for though - the models seem to train pretty quickly, and training/validation loss plateaus:

image

We're going to try poking at this result in a few different ways next, including trying dropout as the regularization axis, and varying/fixing the learning rate to see if different learning rates lead to more distinct regularization/early stopping effects.

review-notebook-app[bot] commented 1 year ago

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

jjc2718 commented 1 year ago

Looks like your models are too good at training, would it be worth trying a harder problem like CCLE -> TCGA to see if you can get more of an observable effect?

Yeah, that might be worth it. I think we're going to start by looking at other types of regularization like dropout and weight decay, but I'll keep this in mind as a possibility.