search

Atome-FE / llama-node

Believe in AI democratization. llama for nodejs backed by llama-rs, llama.cpp and rwkv.cpp, work locally on your laptop CPU. support llama/alpaca/gpt4all/vicuna/rwkv model.
https://llama-node.vercel.app/
Apache License 2.0
862 stars 62 forks source link

app crashes when input is too long #74

Closed ralyodio closed 1 year ago

ralyodio commented 1 year ago 
thread 'tokio-runtime-worker' panicked at 'called `Result::unwrap()` on an `Err` value: Input too long', packages/llama-cpp/src/llama.rs:99:10
stack backtrace:
   0: rust_begin_unwind
             at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/std/src/panicking.rs:579:5
   1: core::panicking::panic_fmt
             at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/core/src/panicking.rs:64:14
   2: core::result::unwrap_failed
             at /rustc/84c898d65adf2f39a5a98507f1fe0ce10a2b8dbc/library/core/src/result.rs:1750:5
   3: tokio::runtime::task::raw::poll
hlhr202 commented 1 year ago

this means your input token length exceeds the nCtx

ralyodio commented 1 year ago

can we keep it from crashing though?

hlhr202 commented 1 year ago

no, this means you have to slice your input so it wont exceeds the nCtx

hlhr202 commented 1 year ago

https://github.com/Atome-FE/llama-node/pull/75/commits/07bad5e31a2cde06ef1da1a5603c2141a64fd855#diff-cc5dc7ec4285ea9174c6bd1690fc2ec4c67a31ccbb16a7a6b6240a7534f71991 this will be handled soon. I just remove the check for nCtx

hlhr202 commented 1 year ago

fixed in v0.1.5

ralyodio commented 1 year ago

Thanks. The app should never crash because of this library, even if inputs are incorrect it should throw an error I can catch.