Currently we have a very simple Seq2Seq model, which nevertheless seems to be performing well. This model can be strongly improved, through making it more similar to the VAE presented in https://pubs.acs.org/doi/abs/10.1021/acscentsci.7b00572
I'm not sure if we should invest the time into add yet another embedding. However, this can probably be done within a day by using a pre-existing implementation.
Needs:
Teacher forcing
Teacher annealing
(optional) Molecular property predictor
The implementation in moses should work for us. We need to be careful with the KL-divergence, as in their experiments they mostly care about generative (ie sampling performance). It may be useful for our case to use smaller βs (KL divergence contributes less to the overall loss), which will reduce sampling performance but increase reconstruction performance (and hopefully lead to a more meaningful latent space).
Currently we have a very simple Seq2Seq model, which nevertheless seems to be performing well. This model can be strongly improved, through making it more similar to the VAE presented in https://pubs.acs.org/doi/abs/10.1021/acscentsci.7b00572
I'm not sure if we should invest the time into add yet another embedding. However, this can probably be done within a day by using a pre-existing implementation.
Needs:
The implementation in moses should work for us. We need to be careful with the KL-divergence, as in their experiments they mostly care about generative (ie sampling performance). It may be useful for our case to use smaller βs (KL divergence contributes less to the overall loss), which will reduce sampling performance but increase reconstruction performance (and hopefully lead to a more meaningful latent space).