Add GPT-2-climate - Githubissues

huggingface / transformers

🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.

https://huggingface.co/transformers

Apache License 2.0

128.5k stars 25.49k forks source link

Add GPT-2-climate #20747

Open saeedashraf opened 1 year ago

saeedashraf commented 1 year ago

Model description

GPT-2 was pretrained on a climate change-related corpus consisting of over 500 thousand abstracts of top climate scientists' articles from trustable sources covering large temporal and spatial scales. The climate-gpt-2 model could further be used for downstream tasks in the climate change domain, including Classification, Fact-checking, and text generation (climate change-related texts).

paper: https://www.climatechange.ai/papers/neurips2022/27

Open source status

[X] The model implementation is available
[X] The model weights are available

Provide useful links for the implementation

@seashr

manish-p-gupta commented 1 year ago

Hey @saeedashraf are we expecting to work on integrating this in Hugging Face? If so then I'll be interested in helping out.

saeedashraf commented 1 year ago

Hi Manish,

Ye ... we would like to fully integrate this.

On Mon, Jan 9, 2023 at 5:47 PM Manish @.***> wrote:

Hey @saeedashraf https://github.com/saeedashraf are we expecting to work on integrating this in Hugging Face? If so then I'll be interested in helping out.

— Reply to this email directly, view it on GitHub https://github.com/huggingface/transformers/issues/20747#issuecomment-1375935797, or unsubscribe https://github.com/notifications/unsubscribe-auth/AFLUPIJHVKSB5GHO53JETW3WRQ6LTANCNFSM6AAAAAAS47TOKU . You are receiving this because you were mentioned.Message ID: @.***>

manish-p-gupta commented 1 year ago

Okay, so the GPT model itself is available in HuggingFace. Do we wish to incorporate this dataset? or just the training objective?