No new results in this PR, just changes to how I'm running some of the experiments. I've made sure that this PR doesn't change the results of the experiments at all, but things are substantially faster now.
Essentially, the cross-cancer experiments I was running before were filling in entries in a matrix by training a new model for each matrix entry. I realized this didn't make sense - since each row of the matrix trains a model on the same gene/cancer type, we can just train the model once per row, then evaluate it lots of times to fill in that row (prediction/model evaluation is much faster than training).
This is a bit more complicated for the pan-cancer experiments, since we actually need to train different models for some matrix columns depending on which cancer type is held out, but it's possible to cache pre-trained models to help in this case as well.
There are more details about the actual optimizations in the code comments, if you're interested.
No new results in this PR, just changes to how I'm running some of the experiments. I've made sure that this PR doesn't change the results of the experiments at all, but things are substantially faster now.
Essentially, the cross-cancer experiments I was running before were filling in entries in a matrix by training a new model for each matrix entry. I realized this didn't make sense - since each row of the matrix trains a model on the same gene/cancer type, we can just train the model once per row, then evaluate it lots of times to fill in that row (prediction/model evaluation is much faster than training).
This is a bit more complicated for the pan-cancer experiments, since we actually need to train different models for some matrix columns depending on which cancer type is held out, but it's possible to cache pre-trained models to help in this case as well.
There are more details about the actual optimizations in the code comments, if you're interested.