how to implement stop sequence in gpt j

kingoflolz / mesh-transformer-jax

Model parallel transformers in JAX and Haiku

Apache License 2.0

6.29k stars 892 forks source link

how to implement stop sequence in gpt j #199

Closed whoislimshady closed 2 years ago

whoislimshady commented 2 years ago

i have fine tuned gpt j model on a dataset but the when i enter the prompt it correctly generate the answer but keep on generating the prompt answer pair is there any way to tell model to stop the genration after certain character

safeeazeem commented 2 years ago

You can follow this repo for stop sequence. I was able to do it using this repo on a V2-8 TPU. You might have to spin up a new instance for this and make minor changes.

whoislimshady commented 2 years ago

@safeeazeem thanks brother

jingrongchen commented 2 years ago

i have the same problem, how did you fix it?