pepperoni21 / ollama-rs

A Rust library allowing to interact with the Ollama API.
MIT License
478 stars 73 forks source link

Added the `Send` trait bound to all stream types #14

Closed varonroy closed 9 months ago

varonroy commented 9 months ago

The goal of this PR is to allow streams to work from tokio::spawn.

Consider the following example:

async fn stream_from_spawn() {
    tokio::spawn(async move {
        let ollama = Arc::new(Ollama::default());

        let mut stream = Ollama::default()
            .send_chat_messages_stream(ChatMessageRequest::new("mistral".to_string(), vec![]))
            .await
            .unwrap();

        let _next = stream.next().await;
    });
}

Since spawn requires its futures to be Send, the above code will not compile and produce the following error message:

error: future cannot be sent between threads safely
   --> src/main.rs:7:5
    |
7   |     tokio::spawn(async move {
    |     ^^^^^^^^^^^^ future created by async block is not `Send`
    |
    = help: the trait `std::marker::Send` is not implemented for `dyn tokio_stream::Stream<Item = Result<ChatMessageResponse, ()>>`
note: future is not `Send` as this value is used across an await

The simplest solution is to add the Send trait bound to all stream types.

pepperoni21 commented 9 months ago

Thanks for your contribution!

functorism commented 6 months ago

This fix was undone in https://github.com/pepperoni21/ollama-rs/pull/13/files#diff-7cbb70dd16d7a45e6e2ff69fa40768ff25c2e6a55aa17810ac365c566dbbe645R11-R14

This would be closed again by https://github.com/pepperoni21/ollama-rs/pull/34