googleapis / python-firestore

Apache License 2.0
214 stars 74 forks source link

Retrieving document snapshots concurrently from different threads leads to a deadlock #913

Open sergeyle opened 5 months ago

sergeyle commented 5 months ago

Calling get() to retrieve a document snapshot concurrently from different threads reliably leads to a deadlock. This was attempted on an M1 Mac and a GCE Cloud Console Linux.

This was tried on google-cloud-firestore=(2.15.0|2.16.0)

Code to reproduce the bug:

import threading
import firebase_admin
from firebase_admin import credentials, firestore

__app = firebase_admin.initialize_app(credentials.ApplicationDefault())

def query_firestore(docid):
    db = firestore.client(__app)
    coll = db.collection("mycollection")
    try:
        print("About to query firestore")
        doc = coll.document(docid).get().to_dict()
        # The flow never reaches here.
        print(f"Document data: {doc}")
    except Exception as e:
        print(f"Error querying Firestore: {e}")

def start_threads(n, docid):
    threads = []
    for _ in range(n):
        thread = threading.Thread(target=query_firestore, args=(docid,))
        threads.append(thread)
        thread.start()
    # Wait for all threads to complete
    for thread in threads:
        thread.join()

if __name__ == "__main__":
    N = 5  # Number of threads
    docid = "1234"
    start_threads(N, docid)
thomsn commented 4 months ago

So Firestore therefore does not work in Firebase Cloud Functions (python). If you get more than one invocation of your function and work with documents the functions deadlock and timeout. This is pretty major issue with the Firebase platform. I was wondering why I was seeing intermittent timeouts.

edit: was unable to reproduce in Firebase Cloud Functions