search

antimatter15 / alpaca.cpp

Locally run an Instruction-Tuned Chat-Style LLM
MIT License
10.27k stars 916 forks source link

Some werfault error with long requests. #97

Open progressionnetwork opened 1 year ago

progressionnetwork commented 1 year ago

Error appear while trying to process long request like: Summarize this text: DeepMind by Chinchilla AI is a popular choice for a large language model, and it has proven itself to be superior to its competitors. In March of 2022, DeepMind released Chinchilla AI. It functions in a manner analogous to that of other large language models such as GPT-3 (175 parameters), Jurassic-1 (178B parameters), Gopher (280B parameters), and Megatron-Turing NLG (300 parameters) (530B parameters). Nonetheless, Chinchilla AI’s main selling point is that it can be created for the same anticipated cost as Gopher, and yet it employs fewer parameters with more data to provide, on average, 7% more accurate results than Gopher.

aleksusklim commented 1 year ago

Change https://github.com/antimatter15/alpaca.cpp/blob/c5ae5d08a56b82c17ef8121bc01221924576ad28/chat.cpp#L1044-L1047 to

                    char buf[16384] = {0};
                    int n_read;
                    if(params.use_color) printf(ANSI_BOLD ANSI_COLOR_GREEN);
                    if (scanf("%16383[^\n]%n%*c", buf, &n_read) <= 0) {

🤦‍♂️🤦‍♂️🤦‍♂️