More specifically, we used GPT-J (Wang and Komatsuzaki, 2021) to generate texts from in-domain sentences. We investigated the feasibility of this domain-specific text generation technique when either no or limited bilingual in-domain dataset is available to fine-tune a generic machine translation model. Combining this approach with back-translation, we could generate huge amounts of synthetic bilingual in-domain data for both use cases. After fine-tuning our baseline generic machine translation model on the purely synthetic dataset, we observed significant improvements of the translation quality of the in-domain test set.
Many thanks for your efforts! We used your GPT-J in our paper "Domain-Specific Text Generation for Machine Translation". The related scripts can be found here.
More specifically, we used GPT-J (Wang and Komatsuzaki, 2021) to generate texts from in-domain sentences. We investigated the feasibility of this domain-specific text generation technique when either no or limited bilingual in-domain dataset is available to fine-tune a generic machine translation model. Combining this approach with back-translation, we could generate huge amounts of synthetic bilingual in-domain data for both use cases. After fine-tuning our baseline generic machine translation model on the purely synthetic dataset, we observed significant improvements of the translation quality of the in-domain test set.