search

kingoflolz / mesh-transformer-jax

Model parallel transformers in JAX and Haiku
Apache License 2.0
6.26k stars 890 forks source link

How to stop model generating #228

Open jingrongchen opened 2 years ago

jingrongchen commented 2 years ago

I have finetuned my model using my own dataset, but when i run the model, it generates a correct answer but the model keep on generating the prompt answer pair, is there any way to tell model to stop the genration after certain character

greyweb commented 12 months ago

Hi, I am trying to finetune the model on my own dataset as well. But even with GPU memory of 24GB, I had issues loading the model weights into it. What would be the suggestion for the compute hardware for finetuning GPTJ 6B.

I am training with input data with an avg of 660 words.