Trying to generate something coherent

[nikitastaf1996]

I am sorry if this is stupid question. I use Google colab.Here is the code I use.It works.But it tends to produce bullshit.What am I doing wrong?

[tridao]

Hi, thanks for trying our model!

There are a couple of things to keep in mind:

1. The models are not finetuned on instructions, they're just trained to predict the next words. So the models probably won't be able to follow instructions like "please write an essay about birds". A good comparison with GPT-Neo 1.3B.
2. At 1.3B or 2.7B parameters we don't expect models to have amazing abilities yet (e.g., GPT3 davinci, ChatGPT). Again, a good comparison is with a Transformer of similar size (e.g. GPT-Neo 1.3B). Given the promising early results, we're excited about scaling these up and finetuning them with instructions.
3. The default options during benchmarking is to use greedy decoding, which is known to be not great for text quality (it tends to produce repetitive text). You can try top_k or top_p sampling.

As an example, with the prompt "Please write an essay about birds", and these options:

generated_text = model.generate(input_ids=input_ids, max_length=max_length,
                                eos_token_id=tokenizer.eos_token_id,
                                return_dict_in_generate=True, output_scores=True, top_p=0.9, top_k=50)

I'm getting

Please write an essay about birds.
To begin with, learn to identify birds by making a sketch of a bird in your mind. Draw a bird on paper and copy it with a pen on your sketch pad or at a computer.

After studying birds on paper, select 10 new birds from the drawing or copy your drawing with a pen; then look through the book for an interesting bird illustration.
[It then continues with more text]

With GPT-Neo-1.3B and the same prompt, I'm getting

Please write an essay about birds in this course

The course has a focus on the importance of birds in the food cycle, and, as part of that, asks you to write an essay about birds in one of these units! Students work in

[nikitastaf1996]

Thank you.I am too used to ChatGPT producing excellent results.Going to try.You are doing excellent job.