Low validity on moses3 dataset

[charlesxu90]

Dear authors,

I tried to train the model with default parameters on moses3.csv dataset, then generate with the trained model. However, the validity that I achieved is 0.868, which I think is much smaller than 0.994 as you mentioned on the paper.

python train.py --run_name unconditional_moses --data_name moses --num_props 0  
python generate.py --model_weight moses_nocond_12layer.pt --csv_name sampled.smiles --data_name moses     

Do you have any suggestion for solving this problem? Or can you provide any hints on how to improve this validity. Thanks!

[charlesxu90]

BTW, why do you use LSTM for scaffold embedding, rather than a linear embedding with position embedding?

For me, the usage of LSTM here feels very strange. If you can use LSTM for scaffold, why not use for all the inputs?

[RishalAggarwal]

I'm not sure we can point out the problem straight up like this, have you tried using the pretrained weights for generation? LSTM was used to represent the scaffold with a single vector, more for convenience than anything. To take advantage of the transformer architecture we need to use linear and positional embeddings for the main molecule.