Hi there,
I am trying to fine-tune the gpt-j-6b model using my custom dataset.
I am trying to figure out the correct format for my dataset.
Currently, when generating tfrecords, I have tried the following formats:
I applied these formats to the whole dataset. The resulting model produced outputs which seemed to suggest the model was unable to recognize the "<|endoftext|>" or "#####" separator token.
Any information on this would be helpful. Please and thank you
Hi i am also trying to figure out the correct format of it? did you make it.
i want to fine tuning the model to do a chatbot. but i don't know how my data format should be
Hi there, I am trying to fine-tune the gpt-j-6b model using my custom dataset. I am trying to figure out the correct format for my dataset. Currently, when generating tfrecords, I have tried the following formats:
MY_CATEGORY: ASSOCIATED_TEXT MY_CATEGORY: ASSOCIATED_TEXT##### MY_CATEGORY: ASSOCIATED_TEXT<|endoftext|> <|endoftext|>MY_CATEGORY: ASSOCIATED_TEXT<|endoftext|> "<|endoftext|>MY_CATEGORY: ASSOCIATED_TEXT<|endoftext|>"
I applied these formats to the whole dataset. The resulting model produced outputs which seemed to suggest the model was unable to recognize the "<|endoftext|>" or "#####" separator token.
Any information on this would be helpful. Please and thank you