search

kathrinse / be_great

A novel approach for synthesizing tabular data using pretrained large language models
MIT License
254 stars 43 forks source link

How can we use other LMs other than distilgpt2 and gpt2 #26

Closed rklohana closed 1 year ago

unnir commented 1 year ago

Hi,

What kind of models do you want to use?

Travisma2233 commented 1 year ago

I have the same problem. If it is possible to use GPT3 in this model ?

unnir commented 1 year ago

@Travisma2233

In general, replacing the model name within the HuggingFace library in GReaT's initialization is typically all you need. For example:

model = GReaT(llm='EleutherAI/gpt-neo-1.3B', batch_size=32, epochs=25)

However, we cannot guarantee that all models will function correctly within this setup. Please open an issue if you need a partially model or even better make a PR.

As for GPT-3, yes, our algorithm is model-agnostic, meaning it can be used with any generative LLM. However, our framework specifically supports only HuggingFace models. If you wish to use OpenAI's GPT-3, a new implementation will be required.

Travisma2233 commented 1 year ago

@Travisma2233

In general, replacing the model name within the HuggingFace library in GReaT's initialization is typically all you need. For example:

model = GReaT(llm='EleutherAI/gpt-neo-1.3B', batch_size=32, epochs=25)

However, we cannot guarantee that all models will function correctly within this setup. Please open an issue if you need a partially model or even better make a PR.

As for GPT-3, yes, our algorithm is model-agnostic, meaning it can be used with any generative LLM. However, our framework specifically supports only HuggingFace models. If you wish to use OpenAI's GPT-3, a new implementation will be required.

Thanks a lot for your answer. Because my network environment is not good, is there a way to run pytorch_model.bin locally