sobelio / llm-chain

`llm-chain` is a powerful rust crate for building chains in large language models allowing you to summarise text and complete complex tasks
https://llm-chain.xyz
MIT License
1.3k stars 128 forks source link

Problem with map reduce tutorial #211

Closed jacohend closed 9 months ago

jacohend commented 1 year ago

I'm only adding some options to executor, and otherwise using a custom model:

    let opts = options!(
        Model: ModelRef::from_path(model_path),
        ModelType: "llama",
        MaxContextSize:  2048_usize,
        NThreads: 12_usize,
        Temperature: 0.7
    );
    let exec = executor!(llama, opts.clone())?;

I'm getting

thread 'main' panicked at 'Cannot block the current thread from within a runtime. This happens because a function attempted to block the current thread while the thread is being used to drive asynchronous tasks.'

thread 'main' panicked at 'Cannot block the current thread from within a runtime. This happens because a function attempted to block the current thread while the thread is being used to drive asynchronous tasks.',
stack backtrace:
   0: rust_begin_unwind
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/std/src/panicking.rs:593:5
   1: core::panicking::panic_fmt
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/panicking.rs:67:14
   2: core::panicking::panic_display
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/panicking.rs:150:5
   3: core::panicking::panic_str
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/panicking.rs:134:5
   4: core::option::expect_failed
             at /rustc/8ede3aae28fe6e4d52b38157d7bfe0d3bceef225/library/core/src/option.rs:1932:5
   5: <llm_chain_llama::executor::LLamaTokenizer as llm_chain::tokens::Tokenizer>::tokenize_str
   6: <llm_chain_llama::executor::Executor as llm_chain::traits::Executor>::tokens_used
   7: llm_chain::tokens::ExecutorTokenCountExt::split_to_fit
   8: <core::iter::adapters::map::Map<I,F> as core::iter::traits::iterator::Iterator>::try_fold
   9: <alloc::vec::Vec<T> as alloc::vec::spec_from_iter::SpecFromIter<T,I>>::from_iter
  10: core::iter::adapters::try_process
  11: llm_chain::chains::map_reduce::Chain::chunk_documents
  12: <core::pin::Pin<P> as core::future::future::Future>::poll
  13: tokio::runtime::scheduler::current_thread::Context::enter
  14: tokio::runtime::context::scoped::Scoped<T>::set
  15: tokio::runtime::context::set_scheduler
  16: tokio::runtime::scheduler::current_thread::CoreGuard::block_on
  17: tokio::runtime::context::runtime::enter_runtime
  18: tokio::runtime::runtime::Runtime::block_on

Was wondering if I could gain some insight by a rust wizard into what's wrong.