search

greenelab / tybalt

Training and evaluating a variational autoencoder for pan-cancer gene expression data
BSD 3-Clause "New" or "Revised" License
162 stars 62 forks source link

ADAGE Implementation Issues #127

Closed gwaybio closed 6 years ago

gwaybio commented 6 years ago

Initial ADAGE models allowed encoder and decoder weights to vary independently. This resulted in relatively poor performance in comparison to other models in simulated and real data reconstruction tasks.

Recent implementations (#123 , #126) have added an option to tie ADAGE weights together (decoder weights are transposed encoder weights). This improved performance. However, the optimal noise parameter observed in a parameter sweep for the pancanatlas data is noise = 0.

z_parameter_adage_remove_sparsity_tiedweights

Based on previous observations, this should not be the case! We need to look into this in more detail. Currently, my thoughts are either:

  1. the implementation is incorrect and doesn't match https://github.com/greenelab/adage
  2. we do not sweep over enough noise parameters (i.e. the optimal noise is somewhere between 0 and 0.1)
cgreene commented 6 years ago

Outside of small-batch high-learning rate scenarios, this seems relatively compatible with what has previously observed: http://discovery.dartmouth.edu/~cgreene/da-psb2015/parameter_sweep_net100.pdf

gwaybio commented 6 years ago

Outside of small-batch high-learning rate scenarios, this seems relatively compatible with what has previously observed:

interesting...perhaps this is accurate then. Thanks for sending the link @cgreene !

I was also able to find a figure (I created it in July 2015) running ADAGE using the https://github.com/greenelab/adage implementation (I believe it was on bitbucket at the time). This was run on a previous version of pancanatlas data (RTCGAToolbox data freeze version 20150402)

fullpancancrossvalidationplot_100nodes

The figure for the psb2015 paper was run using only breast cancer data correct? It looks quite similar to the pancancer results too. I am no longer too concerned about noise = 0 but it will still be good to match implementations