tursodatabase / libsql

libSQL is a fork of SQLite that is both Open Source, and Open Contributions.
https://turso.tech/libsql
MIT License
9.57k stars 253 forks source link

The stream has expired due to inactivity #985

Closed wyhaya closed 7 months ago

wyhaya commented 8 months ago

Example

#[tokio::main]
async fn main() {
    let db = Database::open_remote("https://...turso.io", "...").unwrap();
    let conn = db.connect().unwrap();

    // Ok
    conn.query("select 0", ()).await.unwrap();

    std::thread::sleep(std::time::Duration::from_secs(15));

    // Err: "{\"message\":\"The stream has expired due to inactivity\",\"code\":\"STREAM_EXPIRED\"}",
    conn.query("select 0", ()).await.unwrap();
}

The application needs to run for a long time, but SQL queries are not called frequently. This error occurs whenever there is a period of inactivity, and it cannot be recovered from. Even if called again, the same result will be obtained.

This is quite common in Web App. What is the best approach (perhaps I missed something)?

  1. Should I call db.connect() to obtain a new connection before each SQL query?
  2. Call ping every few seconds to keep the connection active.
  3. Create a libsql connection pool.

Thanks libSQL.

LucioFranco commented 7 months ago

What is the duration/driver for making the queries? If its an http request, I would create a connection per http request that comes in. If you are sleeping I would do the same thing, if you need to run a bunch of sql requests one right after another then re-use the connection.

juliuslipp commented 7 months ago

I'm having the same issue, but with https://github.com/libsql/go-libsql.

wyhaya commented 7 months ago

Thank you for the suggestion, @LucioFranco, this issue has been resolved.

cameron-chan commented 6 months ago

Is there any other workaround for this? I have the same issue when hosting my db locally with turso dev. This issue does not occur when I replace Turso libsql driver with a sqlite3 driver while keeping the same .db file.

haaawk commented 6 months ago

I'm having the same issue, but with https://github.com/libsql/go-libsql.

This should already be fixed in Go

kevinroleke commented 6 months ago

Same issue in Python libsql.

cameron-chan commented 6 months ago

I am experiencing this issue with 'go' also. I am using https://github.com/tursodatabase/libsql

hieuht-htv commented 6 months ago

+1 typescript

frectonz commented 3 months ago

This bug still exists in the rust library. This issue shouldn't be closed.

wyhaya commented 3 months ago

This is my solution and it works fine for me.

use libsql::{Builder, Connection, Database, Result};
use std::sync::Arc;

#[derive(Clone)]
enum Db {
    Local(Connection),
    Remote(Arc<Database>),
}

impl Db {
    async fn new() -> Result<Self> {
        if remote_database {
            let db = Builder::new_remote("".into(), "".into()).build().await?;
            Ok(Self::Remote(Arc::new(db)))
        } else {
            let conn = Builder::new_local("").build().await?.connect()?;
            Ok(Self::Local(conn))
        }
    }

    fn get(&self) -> Result<Connection> {
        let conn = match self {
            // It will always remain connected unless it encounters a major error.
            Self::Local(conn) => conn.clone(),

            // For TCP-based remote libSQL, we establish a new database connection each time.
            // This is necessary because TCP cannot stay connected forever.
            Self::Remote(db) => db.connect()?,
        };
        Ok(conn)
    }
}
async fn request(db: Db, request: Request) {
    let conn = db.get().unwrap();
    conn.execute_batch("insert into test default values;").await.unwrap();
    conn.query("select 'hello libsql!'", ()).await.unwrap();
}
frectonz commented 3 months ago

Thanks @wyhaya , the fix worked.

DogAndHerDude commented 1 month ago

I'm experiencing this using Go as well. And I'm running turso dev --local-file dev.db It seems I have to ping it every time if I want to make a query.