Fine tune on custom dataset

memray / OpenNMT-kpg-release

Keyphrase Generation

MIT License

217 stars 34 forks source link

Fine tune on custom dataset #47

Closed rafikg closed 2 years ago

rafikg commented 2 years ago

Hi @memray Is there any readme or notebook to fine-tune the model on custom dataset?

memray commented 2 years ago

Please stay tuned as we are preparing for a tutorial at ECIR. A new codebase will be released and it can support fine-tuning better. It supports more pretrained models and datasets. New checkpoints will be released as well. Appreciate your patience!

Rui

ahadda5 commented 2 years ago

Dear Rui, any updates on this.

Many notebooks rely on /data/keyphrase which does not exit in the latest master branch
Would love to see such a demo to fine tune models on our own data?

Regards!

memray commented 2 years ago

Hi @rafikg and @ahadda5 , I uploaded datasets I used for training at huggingface. Following the same json format and config files here, you would be able to train models with your own data. Notebook is not a good way for training since it doesn't work well with multiple GPUs.