search

llm-edge / hal-9100

Edge full-stack LLM platform. Written in Rust
MIT License
371 stars 30 forks source link

Run stuck in queued status #20

Closed sullrich closed 9 months ago

sullrich commented 10 months ago

Hello, I am trying to take the project for a spin with llama.cpp backend (works fine with openai client).

I ran through all of the example steps but it remains queued. Am I missing something? Appreciate any guidance that you can give!

curl -X POST http://localhost:3000/threads/a2f18d0f-7b74-4c51-b1f0-bd8aed9e1250/messages \
-H "Content-Type: application/json" \
-d '{
    "role": "user",
    "content": "I need to solve the equation 3x + 11 = 14. Can you help me?"
}'
{"id":"f2e0f99b-c745-4179-b10c-c975aa695fd8","object":"","created_at":1702997905,"thread_id":"a2f18d0f-7b74-4c51-b1f0-bd8aed9e1250","role":"user","content":[{"type":"text","text":{"value":"I need to solve the equation 3x + 11 = 14. Can you help me?","annotations":[]}}],"assistant_id":"00000000-0000-0000-0000-000000000000","run_id":"00000000-0000-0000-0000-000000000000","file_ids":[],"metadata":null}
curl -X POST http://localhost:3000/threads/a2f18d0f-7b74-4c51-b1f0-bd8aed9e1250/runs \
-H "Content-Type: application/json" \
-d '{
    "assistant_id": "588558a1-998c-4cec-b043-05644b9d8f86",
    "instructions": "Please solve the equation."
}'
{"id":"0156e501-e8fe-413e-a560-ed45b19043bc","object":"","created_at":1702997953,"thread_id":"a2f18d0f-7b74-4c51-b1f0-bd8aed9e1250","assistant_id":"588558a1-998c-4cec-b043-05644b9d8f86","status":"queued","required_action":null,"last_error":null,"expires_at":null,"started_at":null,"cancelled_at":null,"failed_at":null,"completed_at":null,"model":"","instructions":"Please solve the equation.","tools":[],"file_ids":[],"metadata":{}}
curl -X GET http://localhost:3000/threads/a2f18d0f-7b74-4c51-b1f0-bd8aed9e1250/runs/0156e501-e8fe-413e-a560-ed45b19043bc \
-H "Content-Type: application/json"
{"id":"0156e501-e8fe-413e-a560-ed45b19043bc","object":"","created_at":1702997953,"thread_id":"a2f18d0f-7b74-4c51-b1f0-bd8aed9e1250","assistant_id":"588558a1-998c-4cec-b043-05644b9d8f86","status":"queued","required_action":null,"last_error":null,"expires_at":null,"started_at":null,"cancelled_at":null,"failed_at":null,"completed_at":null,"model":"","instructions":"Please solve the equation.","tools":[],"file_ids":[],"metadata":{}}
curl -X GET http://localhost:3000/threads/a2f18d0f-7b74-4c51-b1f0-bd8aed9e1250/runs/0156e501-e8fe-413e-a560-ed45b19043bc -H "Content-Type: application/json"
{"id":"0156e501-e8fe-413e-a560-ed45b19043bc","object":"","created_at":1702997953,"thread_id":"a2f18d0f-7b74-4c51-b1f0-bd8aed9e1250","assistant_id":"588558a1-998c-4cec-b043-05644b9d8f86","status":"queued","required_action":null,"last_error":null,"expires_at":null,"started_at":null,"cancelled_at":null,"failed_at":null,"completed_at":null,"model":"","instructions":"Please solve the equation.","tools":[],"file_ids":[],"metadata":{}}

On the assistant side:

[2023-12-19T14:57:45Z INFO assistants_core::assistants] Creating assistant: Assistant { inner: AssistantObject { id: "", object: "", created_at: 0, name: Some("Math Tutor"), description: None, model: "open-orca/mistral-7b-openorca", instructions: Some("You are a personal math tutor. Write and run code to answer math questions."), tools: [Retrieval(AssistantToolsRetrieval { type: "retrieval" })], file_ids: [], metadata: None }, user_id: "00000000-0000-0000-0000-000000000000" } tool: Retrieval(AssistantToolsRetrieval { type: "retrieval" }) [2023-12-19T14:57:59Z INFO assistants_core::threads] Creating thread for user_id: 00000000-0000-0000-0000-000000000000 [2023-12-19T14:58:25Z INFO assistants_core::messages] Adding message to thread_id: a2f18d0f-7b74-4c51-b1f0-bd8aed9e1250, role: User, user_id: 00000000-0000-0000-0000-000000000000 [2023-12-19T14:59:12Z INFO assistants_core::runs] Running assistant_id: 588558a1-998c-4cec-b043-05644b9d8f86 for thread_id: a2f18d0f-7b74-4c51-b1f0-bd8aed9e1250 [2023-12-19T14:59:12Z INFO assistants_core::runs] Creating run for assistant_id: 588558a1-998c-4cec-b043-05644b9d8f86 [2023-12-19T14:59:12Z INFO assistants_core::runs] Updating run for run_id: 0156e501-e8fe-413e-a560-ed45b19043bc [2023-12-19T14:59:47Z INFO assistants_core::runs] Getting run from database for run_id: 0156e501-e8fe-413e-a560-ed45b19043bc [2023-12-19T15:01:01Z INFO assistants_core::runs] Getting run from database for run_id: 0156e501-e8fe-413e-a560-ed45b19043bc

louis030195 commented 10 months ago

Hey welcome! What command did you run to start the API? Did you run make all? Maybe you didn't run the executor (still figuring out the name to call it, the process that consume Redis queue to do LLM calls etc.)

sullrich commented 10 months ago

I ran make docker and make server. Running make all I can see it fire up! Excellent! Thanks so much!

sullrich commented 10 months ago

Once I dropped in my app, seeing this:

[2023-12-19T21:11:08Z INFO assistants_core::runs] Running assistant_id: 608f0c79-a78a-48e0-8523-75a56c2ff55b for thread_id: 028f7171-3bde-4807-87ea-8a7cd06f86c3 [2023-12-19T21:11:08Z INFO assistants_core::runs] Creating run for assistant_id: 608f0c79-a78a-48e0-8523-75a56c2ff55b Failed to create run in database: error returned from database: insert or update on table "runs" violates foreign key constraint "runs_assistant_id_fkey" [2023-12-19T21:11:08Z ERROR tower_http::trace::on_failure] response failed classification=Status code: 500 Internal Server Error latency=1 ms

This works for OpenAI OK. Any ideas?

louis030195 commented 10 months ago

What exactly did you do, which requests/code? Seems like you tried to create a run for a non existing assistant

sullrich commented 10 months ago

Ahh, yes! Somehow the assistant I created earlier disappeared. Does the database reset after stopping and running again?

louis030195 commented 10 months ago

Yeah the make reboot clean up the db. Check the Makefile to just run the DB, not cleaning up

sullrich commented 10 months ago

Thanks for that! Using go + https://github.com/sashabaranov/go-openai getting this backtrace when creating an assistant:

    assistantReq := openai.AssistantRequest{Model: "gpt-4-1106-preview"}
    assistant, err := client.CreateAssistant(ctx, assistantReq)

called Option::unwrap() on a None value stack backtrace: 0: rust_begin_unwind at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/panicking.rs:597:5 1: core::panicking::panic_fmt at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/panicking.rs:72:14 2: core::panicking::panic at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/panicking.rs:127:5 3: core::option::Option::unwrap at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/option.rs:935:21 4: assistants_api_communication::assistants::create_assistant_handler::{{closure}} at ./assistants-api-communication/src/assistants.rs:21:17 5: <F as axum::handler::Handler<(M,T1,T2),S,B>>::call::{{closure}} at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/axum-0.6.20/src/handler/mod.rs:248:53 6: <core::pin::Pin

as core::future::future::Future>::poll at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/future/future.rs:125:9 7: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 8: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 9: <axum::handler::future::IntoServiceFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/axum-0.6.20/src/macros.rs:42:17 10: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 11: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 12: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 13: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 14: <futures_util::future::try_future::MapOk<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 15: <tower::util::map_response::MapResponseFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 16: <core::pin::Pin

as core::future::future::Future>::poll at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/future/future.rs:125:9 17: <tower::util::oneshot::Oneshot<S,Req> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/util/oneshot.rs:97:38 18: <axum::routing::route::RouteFuture<B,E> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/axum-0.6.20/src/routing/route.rs:161:61 19: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 20: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 21: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 22: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 23: <futures_util::future::try_future::MapOk<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 24: <tower::util::map_response::MapResponseFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 25: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 26: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 27: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 28: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 29: <futures_util::future::try_future::MapErr<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 30: <tower::util::map_err::MapErrFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 31: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 32: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 33: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 34: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 35: <futures_util::future::try_future::MapOk<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 36: <tower::util::map_response::MapResponseFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 37: <core::pin::Pin

as core::future::future::Future>::poll at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/future/future.rs:125:9 38: <tower::util::oneshot::Oneshot<S,Req> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/util/oneshot.rs:97:38 39: <axum::routing::route::RouteFuture<B,E> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/axum-0.6.20/src/routing/route.rs:161:61 40: <tower_http::limit::future::ResponseFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-http-0.4.4/src/limit/future.rs:56:53 41: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 42: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 43: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 44: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 45: <futures_util::future::try_future::MapOk<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 46: <tower::util::map_response::MapResponseFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 47: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 48: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 49: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 50: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 51: <futures_util::future::try_future::MapErr<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 52: <tower::util::map_err::MapErrFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 53: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 54: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 55: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 56: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 57: <futures_util::future::try_future::MapOk<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 58: <tower::util::map_response::MapResponseFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 59: <core::pin::Pin

as core::future::future::Future>::poll at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/future/future.rs:125:9 60: <tower::util::oneshot::Oneshot<S,Req> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/util/oneshot.rs:97:38 61: <axum::routing::route::RouteFuture<B,E> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/axum-0.6.20/src/routing/route.rs:161:61 62: <tower_http::trace::future::ResponseFuture<Fut,C,OnResponseT,OnBodyChunkT,OnEosT,OnFailureT> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-http-0.4.4/src/trace/future.rs:52:43 63: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 64: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 65: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 66: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 67: <futures_util::future::try_future::MapOk<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 68: <tower::util::map_response::MapResponseFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 69: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 70: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 71: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 72: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 73: <futures_util::future::try_future::MapErr<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 74: <tower::util::map_err::MapErrFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 75: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 76: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 77: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 78: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 79: <futures_util::future::try_future::MapOk<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 80: <tower::util::map_response::MapResponseFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 81: <core::pin::Pin

as core::future::future::Future>::poll at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/future/future.rs:125:9 82: <tower::util::oneshot::Oneshot<S,Req> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/util/oneshot.rs:97:38 83: <axum::routing::route::RouteFuture<B,E> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/axum-0.6.20/src/routing/route.rs:161:61 84: <tower_http::cors::ResponseFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-http-0.4.4/src/cors/mod.rs:697:56 85: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 86: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 87: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 88: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 89: <futures_util::future::try_future::MapOk<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 90: <tower::util::map_response::MapResponseFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 91: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 note: Some details are omitted, run with RUST_BACKTRACE=full for a verbose backtrace. thread 'tokio-runtime-worker' panicked at /home/sullrich/assistants/assistants-api-communication/src/assistants.rs:21:47: called Option::unwrap() on a None value stack backtrace: 0: rust_begin_unwind at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/panicking.rs:597:5 1: core::panicking::panic_fmt at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/panicking.rs:72:14 2: core::panicking::panic at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/panicking.rs:127:5 3: core::option::Option::unwrap at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/option.rs:935:21 4: assistants_api_communication::assistants::create_assistant_handler::{{closure}} at ./assistants-api-communication/src/assistants.rs:21:17 5: <F as axum::handler::Handler<(M,T1,T2),S,B>>::call::{{closure}} at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/axum-0.6.20/src/handler/mod.rs:248:53 6: <core::pin::Pin

as core::future::future::Future>::poll at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/future/future.rs:125:9 7: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 8: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 9: <axum::handler::future::IntoServiceFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/axum-0.6.20/src/macros.rs:42:17 10: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 11: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 12: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 13: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 14: <futures_util::future::try_future::MapOk<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 15: <tower::util::map_response::MapResponseFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 16: <core::pin::Pin

as core::future::future::Future>::poll at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/future/future.rs:125:9 17: <tower::util::oneshot::Oneshot<S,Req> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/util/oneshot.rs:97:38 18: <axum::routing::route::RouteFuture<B,E> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/axum-0.6.20/src/routing/route.rs:161:61 19: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 20: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 21: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 22: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 23: <futures_util::future::try_future::MapOk<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 24: <tower::util::map_response::MapResponseFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 25: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 26: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 27: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 28: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 29: <futures_util::future::try_future::MapErr<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 30: <tower::util::map_err::MapErrFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 31: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 32: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 33: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 34: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 35: <futures_util::future::try_future::MapOk<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 36: <tower::util::map_response::MapResponseFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 37: <core::pin::Pin

as core::future::future::Future>::poll at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/future/future.rs:125:9 38: <tower::util::oneshot::Oneshot<S,Req> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/util/oneshot.rs:97:38 39: <axum::routing::route::RouteFuture<B,E> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/axum-0.6.20/src/routing/route.rs:161:61 40: <tower_http::limit::future::ResponseFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-http-0.4.4/src/limit/future.rs:56:53 41: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 42: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 43: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 44: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 45: <futures_util::future::try_future::MapOk<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 46: <tower::util::map_response::MapResponseFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 47: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 48: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 49: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 50: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 51: <futures_util::future::try_future::MapErr<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 52: <tower::util::map_err::MapErrFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 53: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 54: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 55: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 56: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 57: <futures_util::future::try_future::MapOk<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 58: <tower::util::map_response::MapResponseFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 59: <core::pin::Pin

as core::future::future::Future>::poll at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/future/future.rs:125:9 60: <tower::util::oneshot::Oneshot<S,Req> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/util/oneshot.rs:97:38 61: <axum::routing::route::RouteFuture<B,E> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/axum-0.6.20/src/routing/route.rs:161:61 62: <tower_http::trace::future::ResponseFuture<Fut,C,OnResponseT,OnBodyChunkT,OnEosT,OnFailureT> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-http-0.4.4/src/trace/future.rs:52:43 63: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 64: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 65: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 66: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 67: <futures_util::future::try_future::MapOk<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 68: <tower::util::map_response::MapResponseFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 69: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 70: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 71: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 72: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 73: <futures_util::future::try_future::MapErr<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 74: <tower::util::map_err::MapErrFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 75: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 76: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 77: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 78: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 79: <futures_util::future::try_future::MapOk<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 80: <tower::util::map_response::MapResponseFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 81: <core::pin::Pin

as core::future::future::Future>::poll at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/future/future.rs:125:9 82: <tower::util::oneshot::Oneshot<S,Req> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/util/oneshot.rs:97:38 83: <axum::routing::route::RouteFuture<B,E> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/axum-0.6.20/src/routing/route.rs:161:61 84: <tower_http::cors::ResponseFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-http-0.4.4/src/cors/mod.rs:697:56 85: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 86: <futures_util::future::try_future::into_future::IntoFuture as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/try_future/into_future.rs:34:9 87: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/future/future/map.rs:55:37 88: <futures_util::future::future::Map<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 89: <futures_util::future::try_future::MapOk<Fut,F> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.29/src/lib.rs:91:13 90: <tower::util::map_response::MapResponseFuture<F,N> as core::future::future::Future>::poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tower-0.4.13/src/macros.rs:38:17 91: ::try_poll at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.29/src/future.rs:82:9 note: Some details are omitted, run with RUST_BACKTRACE=full for a verbose backtrace.

Does anything stick out?

sullrich commented 10 months ago

This was due to missing Tools and other vars: Model: "gpt-4-1106-preview", Name: &assistantName, Description: &assistantDescription, Instructions: &assistantInstructions, Tools: tools,

Afterwards seeing (but I did set the key):

thread 'main' panicked at /home/sullrich/assistants/assistants-extra/src/openai.rs:255:51: OPENAI_API_KEY must be set: NotPresent stack backtrace: 0: rust_begin_unwind at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/std/src/panicking.rs:597:5 1: core::panicking::panic_fmt at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/panicking.rs:72:14 2: core::result::unwrap_failed at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/result.rs:1652:5 3: core::result::Result<T,E>::expect at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/result.rs:1034:23 4: assistants_extra::openai::call_openai_api_with_messages::{{closure}} at ./assistants-extra/src/openai.rs:255:19 5: assistants_extra::llm::llm::{{closure}} at ./assistants-extra/src/llm.rs:64:10 6: assistants_core::assistant::decide_tool_with_llm::{{closure}} at ./assistants-core/src/assistant.rs:268:6 7: assistants_core::assistant::queue_consumer::{{closure}} at ./assistants-core/src/assistant.rs:396:70 8: run_consumer::main::{{closure}} at ./assistants-core/src/bin/run_consumer.rs:61:41 9: tokio::runtime::park::CachedParkThread::block_on::{{closure}} at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/park.rs:282:63 10: tokio::runtime::coop::with_budget at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/coop.rs:107:5 11: tokio::runtime::coop::budget at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/coop.rs:73:5 12: tokio::runtime::park::CachedParkThread::block_on at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/park.rs:282:31 13: tokio::runtime::context::blocking::BlockingRegionGuard::block_on at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/context/blocking.rs:66:9 14: tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}} at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/scheduler/multi_thread/mod.rs:87:13 15: tokio::runtime::context::runtime::enter_runtime at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/context/runtime.rs:65:16 16: tokio::runtime::scheduler::multi_thread::MultiThread::block_on at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/scheduler/multi_thread/mod.rs:86:9 17: tokio::runtime::runtime::Runtime::block_on at /home/sullrich/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.35.0/src/runtime/runtime.rs:350:45 18: run_consumer::main at ./assistants-core/src/bin/run_consumer.rs:60:5 19: core::ops::function::FnOnce::call_once at /rustc/79e9716c980570bfd1f666e3b16ac583f0168962/library/core/src/ops/function.rs:250:5 note: Some details are omitted, run with RUST_BACKTRACE=full for a verbose backtrace.

louis030195 commented 10 months ago

thx just fixed the no tools use case!

how did you set the openai api key?

sullrich commented 10 months ago

Am setting the API Key via the OpenAI client. The fix for tools worked! It created the assistant OK but then I reach:

[2023-12-20T13:03:34Z INFO assistants_core::assistants] Creating assistant: Assistant { inner: AssistantObject { id: "", object: "", created_at: 0, name: Some("My GPT-4 Assistant"), description: None, model: "gpt-4-1106-preview", instructions: Some("Capable of responding to various queries."), tools: [], file_ids: [], metadata: None }, user_id: "00000000-0000-0000-0000-000000000000" } [2023-12-20T13:03:34Z INFO assistants_core::messages] Adding message to thread_id: d746ea83-f42f-4d8e-a00b-a77d347268a4, role: User, user_id: 00000000-0000-0000-0000-000000000000 [2023-12-20T13:03:34Z ERROR tower_http::trace::on_failure] response failed classification=Status code: 500 Internal Server Error latency=2 ms

louis030195 commented 10 months ago

Interesting I guess I could pass the client api key to server but normally you should give the api key to server through .env

Regarding this error I need more context to understand the problem

Actively working on improving the error logging and error returned to client

Can you pass api key in .env and restart your workflow from scratch?

sullrich commented 10 months ago

Thats the interesting thing here, I am not actually using openai in this case. My chat client supports OpenAI and local LLM which is what I am wiring up now. I'll wait for some enhanced logging and dive back in soon!

louis030195 commented 10 months ago

If you create your assistant with gpt-4-1106-preview it will use openai

sullrich commented 10 months ago

I am setting the model to MODEL_URL="http://10.0.250.161:8080/v1/chat/completions" Should I leave the model blank when creating assistant or pass it the URL above?

louis030195 commented 10 months ago

if you are running a LLM on this IP it's good then you need to pass the model name like 

const assistant = await openai.beta.assistants.create({
    instructions: "You are a weather bot. Use the provided functions to answer questions.",
    model: "mistralai/mixtral-8x7b-instruct",
    name: "Weather Bot",
    tools: [{
        "type": "function",
        "function": {
            "name": "getCurrentWeather",
            "description": "Get the weather in location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": { "type": "string", "description": "The city and state e.g. San Francisco, CA" },
                    "unit": { "type": "string" }
                },
                "required": ["location"]
            }
        }
    }]
});
console.log(JSON.stringify(assistant, null, 2));

thanks for all the feedback!

i'll try to make this more clear

sullrich commented 10 months ago

Happy new year! I am getting back into this project and was able to get further into the process but hitting a blocker on the OPENAI side (I am using a local LLM and not OPENAI in this case):

2024-01-02T19:33:11Z ERROR assistants_extra::llm] Error calling OpenAI API: Deserialization Error: missing field id at line 8 column 1 [2024-01-02T19:33:11Z ERROR assistants_core::executor] Run error: Failed to decide tool: Deserialization Error: missing field id at line 8 column 1 [2024-01-02T19:33:11Z INFO assistants_core::runs] Updating run for run_id: f4bda353-a0a6-4fc8-8cf5-ea03acac0092 [2024-01-02T19:33:11Z ERROR assistants_core::executor] Error: Failed to decide tool: Deserialization Error: missing field id at line 8 column 1 [2024-01-02T19:33:11Z INFO assistants_core::executor] Consuming queue [2024-01-02T19:33:13Z INFO assistants_core::runs] Getting run from database for thread_id: ac0c211b-feae-42a1-815f-7d8fcbea00b7 and run_id: f4bda353-a0a6-4fc8-8cf5-ea03acac0092

My .env:

MODEL_URL="http://10.0.250.174:8080/completion" OPENAI_API_KEY="" MODEL_API_KEY="get it here https://docs.perplexity.ai/docs" DATABASE_URL=postgres://postgres:secret@localhost:5432/mydatabase REDIS_URL=redis://127.0.0.1/ S3_ENDPOINT=http://localhost:9000 S3_ACCESS_KEY=minioadmin S3_SECRET_KEY=minioadmin S3_BUCKET_NAME=mybucket

Since I am using a local LLM it is not requiring a key. Any guidance would be extremely appreciated!

louis030195 commented 10 months ago

@sullrich can you share how you run the LLM (cli ran) and the server? (Also feel free to use code blocks to make it easier to read)

my guesses, the id is a property usually returned by OpenAI API-like Chat Completion, so the problem is either:

  1. the MODEL_URL path does not seem familiar to me, so I suspect you are not using an OpenAI compatible LLM API - e.g. something like that: 
    
curl http://10.0.250.174:8080/completion \
-H "Content-Type: application/json" \
-d '{
"model": "mistralai/mixtralabcd",
"messages": [
  {
    "role": "system",
    "content": "You are a helpful assistant."
  },
  {
    "role": "user",
    "content": "Hello!"
  }
]
}'

Which should return a [Chat Completion](https://platform.openai.com/docs/api-reference/chat/object)
2. the `.env` is not properly loaded by the server ("Error calling OpenAI API" is only called when you use OpenAI LLMs and not open source LLMs)

Examples of LLM APIs that does not support OpenAI API-like:
- ollama
- [llama.cpp server example](https://github.com/ggerganov/llama.cpp/tree/master/examples/server)

Examples of LLM APIs that does support OpenAI API-like:
- [FastChat (good if you have a mac)](https://github.com/stellar-amenities/assistants/tree/main/examples/hello-world-mistral-curl)
- [vLLM (good if you have a modern gpu)](https://docs.vllm.ai/en/latest/getting_started/quickstart.html#openai-compatible-server)
- [Perplexity API](https://github.com/stellar-amenities/assistants/tree/main/examples/hello-world-code-interpreter-mixtral-nodejs)
- Mistral API
- anyscale
- together ai

Let me know if that help!
sullrich commented 10 months ago

I am using llama.cpp's openAI server. Its being called exactly like your code block. The client is the openai go client https://github.com/sashabaranov/go-openai

louis030195 commented 10 months ago

Please provide the CLI, code, requests, etc. you used

louis030195 commented 9 months ago

@sullrich no news?

sullrich commented 9 months ago

Sorry I got pulled away on a different project. Unfortunately started getting different errors and I had made no changes on my end. Hopefully soon I can pull together minimal code to share with you.