search

googleapis / java-spanner

Apache License 2.0
64 stars 116 forks source link

Need a way to specify timeout and retry for transaction. #1900

Closed yuanyuan-gl closed 10 months ago

yuanyuan-gl commented 2 years ago

Currently the entire transaction is continually retied for 24 hours according to https://github.com/googleapis/java-spanner/pull/1009.

I think this could get dataflow jobs to stuck or run for long time. Also if the transaction finally get retried successfully after 23 hours, it doesn't give any error or information for customer to debug the issue from their end.

For example, I have saw one case, transactions have been retried for 24 hours again and again due to following error and dataflow jobs gets stuck due to this : 

Error message from worker: com.google.cloud.spanner.AbortedException: ABORTED: io.grpc.StatusRuntimeException: ABORTED: Transaction was aborted. Idle for over 10 seconds. retry_delay { seconds: 254 nanos: 433062132 }

Does it make sense support custom retry and timeout settings on the transaction retry?

siddharth-anvekar commented 1 year ago

Any latest update on this?

olavloite commented 1 year ago

@yuanyuan-gl @siddharth-anvekar We've added a sample for how you can set a transaction timeout here: https://github.com/googleapis/java-spanner/blob/465df7bad12fbea7dbcf6dbabb1b29d088c42665/samples/snippets/src/main/java/com/example/spanner/TransactionTimeoutExample.java#L49

Does that work for your use case?

arpan14 commented 10 months ago

Please feel free to re-open if issue still persists.