I'm only adding some options to executor, and otherwise using a custom model:
let opts = options!(
Model: ModelRef::from_path(model_path),
ModelType: "llama",
MaxContextSize: 2048_usize,
NThreads: 12_usize,
Temperature: 0.7
);
let exec = executor!(llama, opts.clone())?;
I'm getting
thread 'main' panicked at 'Cannot block the current thread from within a runtime. This happens because a function attempted to block the current thread while the thread is being used to drive asynchronous tasks.'
thread 'main' panicked at 'Cannot block the current thread from within a runtime. This happens because a function attempted to block the current thread while the thread is being used to drive asynchronous tasks.',
stack backtrace:
0: rust_begin_unwind
at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/panicking.rs:593:5
1: core::panicking::panic_fmt
at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/panicking.rs:67:14
2: core::panicking::panic_display
at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/panicking.rs:150:5
3: core::panicking::panic_str
at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/panicking.rs:134:5
4: core::option::expect_failed
at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/option.rs:1932:5
5: <llm_chain_llama::executor::LLamaTokenizer as llm_chain::tokens::Tokenizer>::tokenize_str
6: <llm_chain_llama::executor::Executor as llm_chain::traits::Executor>::tokens_used
7: llm_chain::tokens::ExecutorTokenCountExt::split_to_fit
8: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold
9: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter
10: core::iter::adapters::try_process
11: llm_chain::chains::map_reduce::Chain::chunk_documents
12: <core::pin::Pin<P> as core::future::future::Future>::poll
13: tokio::runtime::scheduler::current_thread::Context::enter
14: tokio::runtime::context::scoped::Scoped<T>::set
15: tokio::runtime::context::set_scheduler
16: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
17: tokio::runtime::context::runtime::enter_runtime
18: tokio::runtime::runtime::Runtime::block_on
Was wondering if I could gain some insight by a rust wizard into what's wrong.
I'm only adding some options to executor, and otherwise using a custom model:
I'm getting
thread 'main' panicked at 'Cannot block the current thread from within a runtime. This happens because a function attempted to block the current thread while the thread is being used to drive asynchronous tasks.'
Was wondering if I could gain some insight by a rust wizard into what's wrong.