search

wengong-jin / icml18-jtnn

Junction Tree Variational Autoencoder for Molecular Graph Generation (ICML 2018)
MIT License
510 stars 190 forks source link

Still can't get the same reconstruct score. #41

Open manurubo opened 5 years ago

manurubo commented 5 years ago

So, I've been trying for a lot of time, changing parameters and with the parameters you indicate, and there is no way I can get the same reconstruct score.

I'm doing what you are saying in the readme.md in molvae, with your dataset, with beta 0.001 because in another issue you recommended me to use it. mkdir pre_model/ CUDA_VISIBLE_DEVICES=0 python pretrain.py --train ../data/zinc/train.txt --vocab ../data/zinc/vocab.txt \ --hidden 450 --depth 3 --latent 56 --batch 40 \ --save_dir pre_model/ mkdir vae_model/ CUDA_VISIBLE_DEVICES=0 python vaetrain.py --train ../data/zinc/train.txt --vocab ../data/zinc/vocab.txt \ --hidden 450 --depth 3 --latent 56 --batch 40 --lr 0.0007 --beta 0.001 \ --model pre_model/model.2 --save_dir vae_model/

And after the reconstruct I get about a 0.50 reconstruct, sometimes even less than 0.50. Even the pretrain is giving me 0.51, so with the vae_train Im getting a worse model. Something is clearly wrong here, I have also noticed that my LOG is almost the same as your LOG, except for the stereo parameter, which I get almost always 100%, and you almost never. ¿May this be a problem? Example of MY LOG ( beta 0.005 version, but the steo problem is the same):

KL: 169.9, Word: 96.41, Topo: 99.56, Assm: 97.72, Steo: 98.75 KL: 173.7, Word: 96.51, Topo: 99.49, Assm: 97.92, Steo: 100.00 KL: 163.7, Word: 96.51, Topo: 99.55, Assm: 97.96, Steo: 100.00 KL: 169.0, Word: 96.51, Topo: 99.60, Assm: 98.21, Steo: 100.00 KL: 173.6, Word: 96.53, Topo: 99.47, Assm: 97.71, Steo: 100.00 KL: 171.7, Word: 96.21, Topo: 99.43, Assm: 97.67, Steo: 100.00 KL: 167.2, Word: 96.57, Topo: 99.60, Assm: 97.95, Steo: 100.00 KL: 163.3, Word: 96.30, Topo: 99.53, Assm: 98.09, Steo: 100.00 KL: 158.2, Word: 96.42, Topo: 99.55, Assm: 98.00, Steo: 100.00 KL: 163.7, Word: 96.08, Topo: 99.58, Assm: 97.73, Steo: 100.00 KL: 170.9, Word: 96.65, Topo: 99.56, Assm: 97.54, Steo: 100.00 KL: 166.8, Word: 96.66, Topo: 99.60, Assm: 97.73, Steo: 100.00 KL: 162.2, Word: 96.54, Topo: 99.57, Assm: 97.78, Steo: 100.00 KL: 162.3, Word: 96.34, Topo: 99.41, Assm: 98.27, Steo: 100.00 KL: 174.2, Word: 96.53, Topo: 99.49, Assm: 97.94, Steo: 100.00 KL: 163.9, Word: 96.61, Topo: 99.56, Assm: 97.62, Steo: 100.00 KL: 158.5, Word: 96.14, Topo: 99.51, Assm: 97.48, Steo: 98.33 KL: 160.5, Word: 96.59, Topo: 99.63, Assm: 97.88, Steo: 100.00 KL: 161.2, Word: 96.65, Topo: 99.42, Assm: 97.53, Steo: 100.00 KL: 168.0, Word: 96.47, Topo: 99.67, Assm: 97.89, Steo: 98.75 KL: 163.3, Word: 96.41, Topo: 99.59, Assm: 97.84, Steo: 100.00 KL: 162.9, Word: 96.35, Topo: 99.55, Assm: 98.02, Steo: 100.00 KL: 166.4, Word: 96.38, Topo: 99.55, Assm: 98.17, Steo: 100.00 KL: 165.0, Word: 96.29, Topo: 99.45, Assm: 97.76, Steo: 97.50 learning rate: 0.000630

Example of your LOG:

KL: 257.8, Word: 95.81, Topo: 99.49, Assm: 97.99, Steo: 98.86, PNorm: 302.06, GNorm: 13.44 KL: 254.2, Word: 95.98, Topo: 99.45, Assm: 97.62, Steo: 98.68, PNorm: 302.19, GNorm: 11.76 KL: 257.2, Word: 95.99, Topo: 99.56, Assm: 97.49, Steo: 98.79, PNorm: 302.31, GNorm: 13.35 KL: 259.4, Word: 96.34, Topo: 99.51, Assm: 97.29, Steo: 98.98, PNorm: 302.42, GNorm: 10.87 KL: 255.3, Word: 96.48, Topo: 99.53, Assm: 97.27, Steo: 98.91, PNorm: 302.54, GNorm: 11.11 KL: 252.4, Word: 96.38, Topo: 99.55, Assm: 97.53, Steo: 99.21, PNorm: 302.66, GNorm: 8.37 KL: 245.6, Word: 96.35, Topo: 99.51, Assm: 97.67, Steo: 98.75, PNorm: 302.79, GNorm: 15.39 KL: 230.5, Word: 96.00, Topo: 99.44, Assm: 97.33, Steo: 99.01, PNorm: 302.91, GNorm: 10.65 KL: 244.1, Word: 96.39, Topo: 99.46, Assm: 97.64, Steo: 97.96, PNorm: 303.04, GNorm: 10.75 KL: 227.7, Word: 96.11, Topo: 99.40, Assm: 97.89, Steo: 98.92, PNorm: 303.16, GNorm: 11.46 KL: 240.7, Word: 95.90, Topo: 99.33, Assm: 97.86, Steo: 99.27, PNorm: 303.27, GNorm: 18.13 KL: 238.0, Word: 96.46, Topo: 99.45, Assm: 97.83, Steo: 98.38, PNorm: 303.41, GNorm: 12.82 KL: 244.2, Word: 95.92, Topo: 99.54, Assm: 97.13, Steo: 98.33, PNorm: 303.53, GNorm: 14.16 KL: 236.6, Word: 96.01, Topo: 99.49, Assm: 97.40, Steo: 98.82, PNorm: 303.66, GNorm: 17.48 KL: 231.8, Word: 96.02, Topo: 99.46, Assm: 98.16, Steo: 98.49, PNorm: 303.78, GNorm: 11.04 KL: 237.2, Word: 96.03, Topo: 99.48, Assm: 97.94, Steo: 100.00, PNorm: 303.92, GNorm: 10.08 KL: 237.3, Word: 96.26, Topo: 99.52, Assm: 97.79, Steo: 98.29, PNorm: 304.04, GNorm: 10.13 KL: 230.9, Word: 96.23, Topo: 99.48, Assm: 97.89, Steo: 98.86, PNorm: 304.15, GNorm: 18.26 KL: 243.0, Word: 96.64, Topo: 99.47, Assm: 97.28, Steo: 98.40, PNorm: 304.29, GNorm: 11.93 KL: 233.6, Word: 95.55, Topo: 99.48, Assm: 97.39, Steo: 99.02, PNorm: 304.41, GNorm: 11.69 KL: 228.5, Word: 96.29, Topo: 99.43, Assm: 97.80, Steo: 99.19, PNorm: 304.54, GNorm: 10.17 KL: 225.2, Word: 96.31, Topo: 99.56, Assm: 97.44, Steo: 98.37, PNorm: 304.68, GNorm: 11.25 KL: 232.7, Word: 96.05, Topo: 99.57, Assm: 97.30, Steo: 98.11, PNorm: 304.82, GNorm: 7.84 KL: 226.2, Word: 96.54, Topo: 99.52, Assm: 97.36, Steo: 98.86, PNorm: 304.95, GNorm: 12.34 learning rate: 0.000630

Also, in the pretrain.py and vaetrain.py, there is no way of obtaining the Pnorm and Gnorm as your LOG shows.

It has been frustating for a long time, I dont know what else I can try.

asawanggaa commented 5 years ago

you may change your beta date to 0.004 or less. I use a beta 0.004 version get about a 0.7 reconstruct.