why the deleteonly and the delete_retrieve mode get the same inference result

rpryzant / delete_retrieve_generate

PyTorch implementation of the Delete, Retrieve Generate style transfer algorithm

MIT License

132 stars 26 forks source link

why the deleteonly and the delete_retrieve mode get the same inference result #27

Closed zhunipingan closed 3 years ago

zhunipingan commented 3 years ago

Hi, Thanks for your sharing. In my view, according to the config.json, the model can only do transfer style A to style B. If I want to transfer style B to style A, I should modify the src and tgt setting. If my understanding is right? If I want to change the model mode, I should modify the "model_mode" from "delete" to "delete_retrieve". I do in the above way, but I find the result of delete mode is same as the result of delete retrieve mode(I means the golds file. I think the golds file contains the generated result of the model). I can not find the reason, can you give me some help, thanks.

zhunipingan commented 3 years ago

And what is the differences between the vocab.attribute and ngram.15.attribute. In the yelp_config.json, only ngram.15.attribute is used.

naveen-kinnal commented 3 years ago

@zhunipingan may I know what was your BLEU score for both ? Was it YELP dataset?

zhunipingan commented 3 years ago

@zhunipingan may I know what was your BLEU score for both ? Was it YELP dataset?

I don't know how to calculate the BLEU. During the training, the value of 'METRIC' is about 14. But when I turn to my own dataset, the metric is near to 0. Is there something wrong?

naveen-kinnal commented 3 years ago

@zhunipingan Could you please write me on iamkinnal@gmail.com. I would love to see your data and setup.

rpryzant commented 3 years ago

I'd recommend looking at those two attribute files. The vocab.attribute file was generated using make_attribute_vocab.py, which only considers a unigram vocabulary. The ngram file was generated using make_ngram_attribute_vocab.py which uses ngram vocabularies (up to 4-grams).
Is your metric still near 0? What do the outputs look like? I would recommend adjusting your hyperparameters to try and achieve better model fit.

rpryzant commented 3 years ago

Closing, ~1 week since last activity.