Open hytxx opened 1 month ago
hytxx
commented
1 month ago I'm probably guessing that the error was reported because of an issue with downloading the model, since I'm deploying in an intranet environment and don't have access to the web, I manually downloaded the required model, but I don't know why it's still going to download it, is there some step I'm missing?
wsxiaoys
commented
1 month ago Hi - for air gapped deployment, please refer https://tabby.tabbyml.com/blog/2024/03/25/deploy-tabby-in-air-gapped-environment-with-docker/
hytxx
commented
1 month ago Hi - for air gapped deployment, please refer https://tabby.tabbyml.com/blog/2024/03/25/deploy-tabby-in-air-gapped-environment-with-docker/您好 - 对于气隙部署，请参阅 https://tabby.tabbyml.com/blog/2024/03/25/deploy-tabby-in-air-gapped-environment-with-docker/
`DEPRECATED: The legacy builder is deprecated and will be removed in a future release.
Install the buildx component to build images with BuildKit:
https://docs.docker.com/go/buildx/Sending build context to Docker daemon 2.048kB
Step 1/3 : FROM tabbyml/tabby
---> 769a715d2734
Step 2/3 : ENV TABBY_MODEL_CACHE_ROOT=/models
---> Running in 9270f44db966
Removing intermediate container 9270f44db966
---> 4e76ce942890
Step 3/3 : RUN /opt/tabby/bin/tabby-cpu download --model StarCoder-1B
---> Running in 3d323eda316e
The application panicked (crashed).
Message: Failed to fetch model organization
Caused by:
0: error sending request for url (https://raw.githubusercontent.com/TabbyML/registry-tabby/main/models.json)
1: client error (Connect)
2: tcp connect error: Cannot assign requested address (os error 99)
3: Cannot assign requested address (os error 99)Location: /root/workspace/crates/tabby-common/src/registry.rs:62
Backtrace omitted. Run with RUST_BACKTRACE=1 environment variable to display it.
Run with RUST_BACKTRACE=full to include source snippets.
The command '/bin/sh -c /opt/tabby/bin/tabby-cpu download --model StarCoder-1B' returned a non-zero code: 101
`
Because of the region problem, I can't download the model directly, follow my steps above, download the model manually and then specify the model path, why doesn't this work?
hytxx
commented
1 month ago Hi - for air gapped deployment, please refer https://tabby.tabbyml.com/blog/2024/03/25/deploy-tabby-in-air-gapped-environment-with-docker/您好 - 对于气隙部署，请参阅 https://tabby.tabbyml.com/blog/2024/03/25/deploy-tabby-in-air-gapped-environment-with-docker/
but got error:
2024-07-11T07:35:33.805755Z INFO tabby::services::model: crates/tabby/src/services/model/mod.rs:44: Loading model from local path /data/models/TabbyML/DeepseekCoder-6.7B
The application panicked (crashed).
Message: Failed to fetch model organization: Failed to download
Caused by:
Stack backtrace:
0: tabby_common::registry::ModelRegistry::new::{{closure}}
1: tabby_download::download_model::{{closure}}
2: tabby::services::model::download_model_if_needed::{{closure}}
3: tabby::main::{{closure}}
4: tokio::runtime::park::CachedParkThread::block_on
5: tokio::runtime::context::runtime::enter_runtime
6: tokio::runtime::runtime::Runtime::block_on
7: tabby::main
8: std::sys_common::backtrace::__rust_begin_short_backtrace
9: std::rt::lang_start::{{closure}}
10: core::ops::function::impls::<impl core::ops::function::FnOnce for &F>::call_once
11: std::panicking::try::do_call
12: std::panicking::try
13: std::panic::catch_unwind
14: std::rt::lang_start_internal::{{closure}}
15: std::panicking::try::do_call
16: std::panicking::try
17: std::panic::catch_unwind
18: std::rt::lang_start_internal
19: main
20:
21: __libc_start_main
22: _start
Location: /root/workspace/crates/tabby-common/src/registry.rs:62
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
9: tabby_common::registry::ModelRegistry::new::{{closure}}::h9fc5b5910c59e2d4
10: tabby_download::download_model::{{closure}}::h2afaf8f50140f7e0
11: tabby::services::model::download_model_if_needed::{{closure}}::h7a37e0f73e254e39
12: tabby::main::{{closure}}::h2bdc8533ce6aacb4
13: tokio::runtime::park::CachedParkThread::block_on::he18aa8b73cf5abd3
14: tokio::runtime::context::runtime::enter_runtime::h71d9772cbcdeefc6
15: tokio::runtime::runtime::Runtime::block_on::h2c17fbddcce4400e
16: tabby::main::hece9075b72caf8e4
17: std::sys_common::backtrace::__rust_begin_short_backtrace::he1d35793bc872d1d
18: std::rt::lang_start::{{closure}}::h7bcf0ff8d5702892
19: core::ops::function::impls::<impl core::ops::function::FnOnce for &F>::call_once::h37600b1e5eea4ecd
20: std::panicking::try::do_call::hb4bda49fa13a0c2b
21: std::panicking::try::h8bbf75149211aaaa
22: std::panic::catch_unwind::h8c78ec68ebea34cb
23: std::rt::lang_start_internal::{{closure}}::hffdf44a19fd9e220
24: std::panicking::try::do_call::hcb3194972c74716d
25: std::panicking::try::hcdc6892c5f0dba4c
26: std::panic::catch_unwind::h4910beb4573f4776
27: std::rt::lang_start_internal::h6939038e2873596b
28: main
29: __libc_start_main
30: _start
Run with COLORBT_SHOW_HIDDEN=1 environment variable to disable frame filtering.
then I don't know what to do next.