Service Not Ready - Transport Error When Fetching Data

[dikkadev]

I am encountering an issue when trying to fetch data. The issue occurs, no matter what operation I try to perform. Even simple operations like listing all documents result in the same error.

Code

pub async fn get_one(db: &FirestoreDb, team: &str, id: &str) -> Option<Snippet> {
    db.fluent()
        .select()
        .by_id_in(team)
        .obj()
        .one(id)
        .await
        .unwrap()
}

Error

DatabaseError(FirestoreDatabaseError { public: FirestoreErrorPublicGenericDetails { code: "Unknown" }, details: "status: Unknown, message: \"Service was not ready: transport error\", details: [], metadata: MetadataMap { headers: {} }", retry_possible: false })

[abdolence]

Hm, interesting. Can you describe a bit how do you instantiate the Firestore object? Does it work when you just create a new instance and run any query on it?

[abdolence]

Also features and versions for tonic/tokio in your Cargo files (lock file if you use it) might help.

[dikkadev]

My original connection code is this:

static Connection: OnceLock<FirestoreDb> = OnceLock::new();

pub fn connection() -> &'static FirestoreDb {
    thread::spawn(|| {
        Connection.get_or_init(|| {
            let runtime = match Runtime::new() {
                Ok(runtime) => runtime,
                Err(_) => {
                    eprintln!("Failed to create a new Runtime");
                    std::process::exit(1);
                }
            };

            runtime.block_on(async {
                let gcp_project = match std::env::var("GCP_PROJECT") {
                    Ok(project) => project,
                    Err(_) => {
                        eprintln!("Failed to get GCP_PROJECT from environment variables");
                        std::process::exit(1);
                    }
                };
                match FirestoreDb::new(gcp_project).await {
                    Ok(db) => db,
                    Err(e) => {
                        eprintln!("Failed connection to Firestore: {}", e);
                        std::process::exit(1);
                    }
                }
            })
        })
    })
    .join()
    .unwrap_or_else(|_| {
        eprintln!("Failed to join the thread");
        std::process::exit(1);
    })
}

I've just played around with this a bit. If I do the query inside the same runtime.block_on everything works fine. If I do it afterward in a new runtime.block_on it does not work. However, when looking at the db with a println debug, it seems to be the same inside the runtime.block_on and after it is returned.

I need to use a new thread and runtime.block_on because you can't make the get_or_init async as far as I know. Although I do not necessarily need the connection to be in a OnceCell or OnceLock it would be nice.

[abdolence]

I might be wrong, but with this approach I think you're observing an issue with gRPC that depends on its original runtime in which it was created. This is more like Tonic implementation restriction, so if this is the case, I can't fix it in the library.

There is some discussion here, maybe related to this: https://github.com/hyperium/tonic/issues/942

[abdolence]

Did you try to reuse that runtime you creating inside? Maybe you can create on more static runtime or something?

[dikkadev]

Just played around a bit more, it definitely is the runtime issue. Unfortunately, I haven't found a way to make a static Runtime, so I'll probably end up just passing the connection (and maybe runtime) around as parameters.

Thanks for the quick help!

[abdolence]

Let me know if you find a good way to resolve this with Tonic/Tokio.

[abdolence]

static DB: tokio::sync::OnceCell<FirestoreDb> = tokio::sync::OnceCell::const_new();
static DB_RUNTIME: once_cell::sync::OnceCell<tokio::runtime::Runtime> =
    once_cell::sync::OnceCell::new();

fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
    let rt = DB_RUNTIME.get_or_init(|| tokio::runtime::Runtime::new().unwrap());
    rt.block_on(async move {
        let db = DB
            .get_or_init(|| async { FirestoreDb::new("<project-id>".to_string()).await.unwrap() })
            .await;

        db.ping().await.unwrap();
    });

    rt.block_on(async move {
        let db = DB
            .get_or_init(|| async { FirestoreDb::new("<project-id>".to_string()).await.unwrap() })
            .await;

        db.ping().await.unwrap();
    });
}

Something like this seems working (I skipped obviously error handling etc), but I'm not sure if this anyhow better than propagating some kind of app specific structure with globals instead. I usually do later for async stuff.