The application hangs after 30 minutes for long running PDML

[yuanyuan-gl]

There is an internal bug opened for this and potentially a fix in https://github.com/asthamohta/python-spanner/tree/retry-error. But it is not confirmed yet, so want to open this in github issue to track it and just in case it is not recorded.

No exception thrown up with following code using python google-cloud-spanner==3.19.0 running on GCE instance, the script just hangs after 30 minutes and the deletion stopped after 30 minutes

from google.cloud import spanner
from google.cloud.spanner_v1 import RequestOptions
import sys

def run(query: str, instance_id: str, database_id: str, project: str):
    try:
        spanner_client = spanner.Client(project=project)
        database = spanner_client.instance(instance_id).database(database_id)
        print(f"Connected to database {instance_id}/{database_id}")
        print(f"Running query: {query}")
        request_options = RequestOptions({"priority": RequestOptions.Priority.PRIORITY_HIGH,"transaction_tag": "delete transaction tag","request_tag": "delete request tag"})
        row_ct = database.execute_partitioned_dml(query, request_options=request_options)
        print(f"{row_ct} records updated.")
    except Exception as e:
        print(e)

if __name__ == "__main__":
    query = sys.argv[4]
    project = sys.argv[1]
    instance = sys.argv[2]
    database = sys.argv[3]
    run(query, instance, database, project)

[yuanyuan-gl]

Just clarify, the script just hangs after 30 minutes and the deletion also stops after 30 minutes.

[anpjai]

Is there an update on this. I am trying to delete 200m rows and seems to get different results. some times it succeeds. more than often it fails with timeout or deadline exceeded error.

[dakshinrajavel]

I face the similar issue when I run a plain SQL from terminal gcloud spanner databases execute-sql XXXX \ --instance=my-dev-instance \ --sql="delete * from mytable where id ='1234';"
This involves deleting 300million rows. I tried running this on nohup as well. No error, but I don't see the delete happening after some 30 minutes. When are we expecting a fix for this

[Hackeron]

Hello @asthamohta, I am also facing the similar issue.

Is there any timeline for this issue to get resolved? I see this issue is open since long time and with no action.