Open HeroSong666 opened 6 months ago
Or, I noticed that in https://huggingface.co/bigcode/starcoder2-3b The inference API can generate code piece by piece, each time I press the Compute. How can I implement such functionality? (For example, in python, every time I send a request, the model returns me a portion of the results. The next time I send a request, it will send the request based on the previous request + previous results it returns. In this way, the code can be completed step by step without creating redundant parts.) Many thanks for your advice!
I am new to starcode.
when I run the follow demo:
it returns:
it doesn`t finish. so I SET the max_length=120, then it returns:
The part
is redundant. now my solution is:
But I don`t think it a good idea. I want the model to return the results in one go without generating redundant parts. How can I do that? Could you give me some advice?