search

floneum / floneum

Instant, controllable, local pre-trained AI models in Rust
http://floneum.com/kalosm
Apache License 2.0
1.42k stars 70 forks source link

Unable to download models : Unexpected status code: 401 #280

Open dbkblk opened 2 weeks ago

dbkblk commented 2 weeks ago

Problem

No source are working on kalosm::language: error Unexpected status code: 401

Steps To Reproduce

Example code:

use kalosm::language::*;

#[tokio::main]
async fn main() -> Result<(), anyhow::Error> {
    let model = Llama::builder()
        .with_source(LlamaSource::mistral_7b_instruct_2())
        .build()
        .await
        .unwrap();

    let text = model
        .generate_text("The capital of France is")
        .with_max_length(300)
        .await
        .unwrap();
    println!("{}", text);

    Ok(())
}

Output:

RUST_BACKTRACE=1 cargo run
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.18s
warning: the following packages contain code that will be rejected by a future version of Rust: nom v1.2.4, xml5ever v0.16.2
note: to see what the problems were, use the option `--future-incompat-report`, or run `cargo report future-incompatibilities --id 1`
     Running `target/debug/kalosm_llm`
Running on CPU, to run on GPU, build with `--features cuda`
⠁ [00:00:00] [░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░] (0/100, ETA 0s)
⠁ [00:00:00] [░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░] (0/100, ETA 0s)                                                                                                                                                                               thread 'main' panicked at src/main.rs:11:10:
called `Result::unwrap()` on an `Err` value: Unexpected status code: 401

Stack backtrace:
   0: anyhow::error::<impl anyhow::Error>::msg
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.89/src/backtrace.rs:27:14
   1: kalosm_common::cache::download_into::{{closure}}
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/kalosm-common-0.3.3/src/cache.rs:287:9
   2: kalosm_common::cache::Cache::get::{{closure}}
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/kalosm-common-0.3.3/src/cache.rs:212:18
   3: kalosm_llama::source::LlamaSource::tokenizer::{{closure}}
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/kalosm-llama-0.3.3/src/source.rs:75:72
   4: kalosm_llama::LlamaBuilder::build_with_loading_handler::{{closure}}
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/kalosm-llama-0.3.3/src/lib.rs:283:18
   5: kalosm_llama::LlamaBuilder::build::{{closure}}
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/kalosm-llama-0.3.3/src/lib.rs:314:14
   6: kalosm_llm::main::{{closure}}
             at ./src/main.rs:10:10
   7: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/future/future.rs:123:9
   8: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/park.rs:281:63
   9: tokio::runtime::coop::with_budget
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/coop.rs:107:5
  10: tokio::runtime::coop::budget
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/coop.rs:73:5
  11: tokio::runtime::park::CachedParkThread::block_on
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/park.rs:281:31
  12: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/context/blocking.rs:66:9
  13: tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/scheduler/multi_thread/mod.rs:87:13
  14: tokio::runtime::context::runtime::enter_runtime
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/context/runtime.rs:65:16
  15: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/scheduler/multi_thread/mod.rs:86:9
  16: tokio::runtime::runtime::Runtime::block_on_inner
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/runtime.rs:363:45
  17: tokio::runtime::runtime::Runtime::block_on
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/runtime.rs:333:13
  18: kalosm_llm::main
             at ./src/main.rs:23:5
  19: core::ops::function::FnOnce::call_once
             at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/ops/function.rs:250:5
  20: std::sys::backtrace::__rust_begin_short_backtrace
             at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/sys/backtrace.rs:152:18
  21: std::rt::lang_start::{{closure}}
             at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/rt.rs:162:18
  22: core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &F>::call_once
             at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/ops/function.rs:284:13
  23: std::panicking::try::do_call
             at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/panicking.rs:557:40
  24: std::panicking::try
             at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/panicking.rs:521:19
  25: std::panic::catch_unwind
             at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/panic.rs:350:14
  26: std::rt::lang_start_internal::{{closure}}
             at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/rt.rs:141:48
  27: std::panicking::try::do_call
             at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/panicking.rs:557:40
  28: std::panicking::try
             at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/panicking.rs:521:19
  29: std::panic::catch_unwind
             at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/panic.rs:350:14
  30: std::rt::lang_start_internal
             at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/rt.rs:141:20
  31: std::rt::lang_start
             at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/rt.rs:161:17
  32: main
  33: <unknown>
  34: __libc_start_main
  35: _start
⠁ [00:00:00] [░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░] (0/100, ETA 0s)
⠙ [00:00:00] [░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░] (0/100, ETA 0s)                                                                                                                                                                                  0: rust_begin_unwind
             at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/std/src/panicking.rs:665:5
   1: core::panicking::panic_fmt
             at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/panicking.rs:74:14
   2: core::result::unwrap_failed
             at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/result.rs:1679:5
   3: core::result::Result<T,E>::unwrap
             at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/result.rs:1102:23
   4: kalosm_llm::main::{{closure}}
             at ./src/main.rs:5:17
   5: <core::pin::Pin<P> as core::future::future::Future>::poll
             at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/future/future.rs:123:9
   6: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/park.rs:281:63
   7: tokio::runtime::coop::with_budget
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/coop.rs:107:5
   8: tokio::runtime::coop::budget
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/coop.rs:73:5
   9: tokio::runtime::park::CachedParkThread::block_on
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/park.rs:281:31
  10: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/context/blocking.rs:66:9
  11: tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/scheduler/multi_thread/mod.rs:87:13
  12: tokio::runtime::context::runtime::enter_runtime
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/context/runtime.rs:65:16
  13: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/scheduler/multi_thread/mod.rs:86:9
  14: tokio::runtime::runtime::Runtime::block_on_inner
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/runtime.rs:363:45
  15: tokio::runtime::runtime::Runtime::block_on
             at /home/hadrien/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.40.0/src/runtime/runtime.rs:333:13
  16: kalosm_llm::main
             at ./src/main.rs:23:5
  17: core::ops::function::FnOnce::call_once
             at /rustc/eeb90cda1969383f56a2637cbd3037bdf598841c/library/core/src/ops/function.rs:250:5

Expected behavior

The example is supposed to download the model and execute it.

Environment:

Questionnaire

ealmloff commented 2 weeks ago

That model is gated on huggingface which means you need to go to the page and request access. Make sure you have done that and you have set the huggingface token. You can set it with huggingface-cli login, or the environment variable HF_TOKEN. Or set it manually with https://docs.rs/kalosm-common/0.3.2/kalosm_common/struct.Cache.html#method.with_huggingface_token

dbkblk commented 1 week ago

Thank you for your answer, it worked to pass the 401 error. I would suggest to add this information in this documentation page: https://docs.rs/kalosm/latest/kalosm/language/struct.LlamaSource.html Maybe it's just me, but I missed it when I read the doc.

Now, NeuralHermes-2.5-Mistral-7B-GGUF return 403 even though there's no authorization required, but that's another issue.