TabbyML for docker-compose throws error

[PeterTucker]

Describe the bug Tabbyml for docker-compose throws error: tabby1-1 | Writing to new file. tabby1-1 | thread 'main' panicked at /root/workspace/crates/aim-downloader/src/https.rs:150:70: tabby1-1 | called Result::unwrap() on an Err value: "Error while downloading." tabby1-1 | stack backtrace: tabby1-1 | 0: 0x556e32756676 - std::backtrace_rs::backtrace::libunwind::trace::hbee8a7973eeb6c93 tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/../../backtrace/src/backtrace/libunwind.rs:104:5 tabby1-1 | 1: 0x556e32756676 - std::backtrace_rs::backtrace::trace_unsynchronized::hc8ac75eea3aa6899 tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5 tabby1-1 | 2: 0x556e32756676 - std::sys_common::backtrace::_print_fmt::hc7f3e3b5298b1083 tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:68:5 tabby1-1 | 3: 0x556e32756676 - ::fmt::hbb235daedd7c6190 tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:44:22 tabby1-1 | 4: 0x556e327874c0 - core::fmt::rt::Argument::fmt::h76c38a80d925a410 tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/fmt/rt.rs:142:9 tabby1-1 | 5: 0x556e327874c0 - core::fmt::write::h3ed6aeaa977c8e45 tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/fmt/mod.rs:1120:17 tabby1-1 | 6: 0x556e3275311f - std::io::Write::write_fmt::h78b18af5775fedb5 tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/io/mod.rs:1810:15 tabby1-1 | 7: 0x556e32756454 - std::sys_common::backtrace::_print::h5d645a07e0fcfdbb tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:47:5 tabby1-1 | 8: 0x556e32756454 - std::sys_common::backtrace::print::h85035a511aafe7a8 tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:34:9 tabby1-1 | 9: 0x556e32757cd7 - std::panicking::default_hook::{{closure}}::hcce8cea212785a25 tabby1-1 | 10: 0x556e32757a39 - std::panicking::default_hook::hf5fcb0f213fe709a tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:292:9 tabby1-1 | 11: 0x556e32758168 - std::panicking::rust_panic_with_hook::h095fccf1dc9379ee tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:779:13 tabby1-1 | 12: 0x556e32758042 - std::panicking::begin_panic_handler::{{closure}}::h032ba12139b353db tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:657:13 tabby1-1 | 13: 0x556e32756b76 - std::sys_common::backtrace::rust_end_short_backtrace::h9259bc2ff8fd0f76 tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/sys_common/backtrace.rs:171:18 tabby1-1 | 14: 0x556e32757da0 - rust_begin_unwind tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:645:5 tabby1-1 | 15: 0x556e31071345 - core::panicking::panic_fmt::h784f20a50eaab275 tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/panicking.rs:72:14 tabby1-1 | 16: 0x556e31071893 - core::result::unwrap_failed::h03d8a5018196e1cd tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/result.rs:1649:5 tabby1-1 | 17: 0x556e310efff9 - tabby_download::download_file::{{closure}}::hf4a000fa739be69e tabby1-1 | 18: 0x556e311237a0 - <tokio_retry::future::RetryIf<I,A,C> as core::future::future::Future>::poll::h911c3d7385f5f568 tabby1-1 | 19: 0x556e314a1584 - tabby_download::download_model::{{closure}}::hd5c83640a8fcf6a5 tabby1-1 | 20: 0x556e314df574 - tabby::services::model::download_model_if_needed::{{closure}}::h7aa6b484e3614d23 tabby1-1 | 21: 0x556e314e2a83 - tabby::main::{{closure}}::h5871e0ee9ff0a5e5 tabby1-1 | 22: 0x556e314cff39 - tokio::runtime::park::CachedParkThread::block_on::ha5ad3e7aadffe765 tabby1-1 | 23: 0x556e31135435 - tabby::main::h8c292e46bcce4562 tabby1-1 | 24: 0x556e31266da3 - std::sys_common::backtrace::rust_begin_short_backtrace::h053326dcb89d797e tabby1-1 | 25: 0x556e31221e39 - std::rt::lang_start::{{closure}}::h6f3dbe64c8a6df18 tabby1-1 | 26: 0x556e32749a21 - core::ops::function::impls::<impl core::ops::function::FnOnce for &F>::call_once::h37600b1e5eea4ecd tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/ops/function.rs:284:13 tabby1-1 | 27: 0x556e32749a21 - std::panicking::try::do_call::hb4bda49fa13a0c2b tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40 tabby1-1 | 28: 0x556e32749a21 - std::panicking::try::h8bbf75149211aaaa tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19 tabby1-1 | 29: 0x556e32749a21 - std::panic::catch_unwind::h8c78ec68ebea34cb tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14 tabby1-1 | 30: 0x556e32749a21 - std::rt::lang_start_internal::{{closure}}::hffdf44a19fd9e220 tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:48 tabby1-1 | 31: 0x556e32749a21 - std::panicking::try::do_call::hcb3194972c74716d tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:552:40 tabby1-1 | 32: 0x556e32749a21 - std::panicking::try::hcdc6892c5f0dba4c tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:516:19 tabby1-1 | 33: 0x556e32749a21 - std::panic::catch_unwind::h4910beb4573f4776 tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panic.rs:142:14 tabby1-1 | 34: 0x556e32749a21 - std::rt::lang_start_internal::h6939038e2873596b tabby1-1 | at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/rt.rs:148:20 tabby1-1 | 35: 0x556e31136195 - main tabby1-1 | 36: 0x7fedce1bfd90 - tabby1-1 | 37: 0x7fedce1bfe40 - __libc_start_main tabby1-1 | 38: 0x556e31081d85 - _start tabby1-1 | 39: 0x0 - tabby1-1 exited with code 0

version: '3.8'

services:
  tabby1:
    restart: always
    image: tabbyml/tabby
    command: serve --model TabbyML/StarCoder-1B --chat-model TabbyML/WizardCoder-3B --device cuda --webserver
    volumes:
      - ./data:/data
    ports:
      - 8080:8080
    environment:
      - RUST_BACKTRACE=full
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

Information about your version Please provide output of tabby --version latest: image: tabbyml/tabby

Could you double check your internet environment has access to github.com and huggingface.com?

[PeterTucker]

Seems to have internet connection.

For some reason when I just download the model, it works fine: serve --model TabbyML/StarCoder2-3B --device cuda --webserver

However, adding a --chat-model seems to break it.

[wsxiaoys]

Please make sure you're using models from https://tabby.tabbyml.com/docs/models/ listed in Chat models section

[PeterTucker]

TabbyML/WizardCoder-3B is a chat model that is on that list.