search

theislab / chemCPA

Code for "Predicting Cellular Responses to Novel Drug Perturbations at a Single-Cell Resolution", NeurIPS 2022.
https://arxiv.org/abs/2204.13545
MIT License
88 stars 23 forks source link

EXP: Experiment with very large models on LINCS #64

Closed siboehm closed 2 years ago

siboehm commented 2 years ago

The Vanilla Model with ~6M parameters is outperforming most of our other model, however these other models have maximum ~2M parameters.

It might be worth scheduling some runs with very large models, to see how they perform.

siboehm commented 2 years ago

Currently running this

siboehm commented 2 years ago

Trained 5 new GROVER models on LINCS, with large encoders (up to depth 7) and large drug embedders and dosers (depth ~4). Latent size was 64, I adjusted the adversarial parameters and only a single model was easy to disentangle (which is good).

There weren't any surprisingly good results, so I think we're in the right ballpark regarding sizes.

In this plot I selected the top 2 models that had a reasonable disentanglement loss (<0.1), and the lowest val score. Batch_id: 45 (large models), 39 (normal models). image

siboehm commented 2 years ago

@MxMstrmn Closing this, unless you have objections