More instructions to train the primary model

[sudoRicheek]

Hi @Sachin19, thanks for your exciting work!

I had some doubts regarding the primary model pretraining. In the paper, it mentioned:

For our primary objective, we use a inverse-paraphrasing model as defined in §3.1, which we train on a corpus of Yelp Reviews6 [45]. First, we paraphrase each sentence in the corpus as described in Krishna et al. [28] creating a pseudo-parallel corpus (of reviews and their paraphrases) and train G as an inverse-paraphrase model to translate the paraphrases back to the original reviews.

Is there any existing code to pretrain this primary model? Can you point us to it?

If not, are there any pretrained models which we can directly use in: https://github.com/Sachin19/mucoco/blob/054fb3c8f207b71cde6ec41b1a01b0bcf89be4b8/examples/style-transfer/decode_example.sh#L25

[liushz]

same question，today is September 9，the question still no answer

[Sachin19]

Hi,

The primary model is the same as the one STRAP uses. The instructions to train are here to (a) paraphrase the data, and (b) fine-tune an inverse paraphrase model.

For training the constraint models:

(1) For huggingface-based models (classifiers), you can follow the instructions here (and dataset and hyperparameter details from the paper). (2) For sentence transformer models (similarity models), you can follow the instructions here (by fine-tuning a GPT2 based model).

Hope this helps!