estuary / connectors

Connectors for capturing data from external data sources
Other
50 stars 17 forks source link

materialize-s3-iceberg: set a timeout on append files #2113

Closed williamhbaker closed 3 weeks ago

williamhbaker commented 3 weeks ago

Description:

We have observed that append requests can hang indefinitely when using a REST catalog. This implements a conservative timeout of 5 minutes for these append requests, so that the connector can log an error and re-start if an append request takes an extremely long time.

Workflow steps:

(How does one use this feature, and how has it changed)

Documentation links affected:

(list any documentation links that you created, or existing ones that you've identified as needing updates, along with a brief description)

Notes for reviewers:

(anything that might help someone review this PR)


This change is Reviewable