iceberg: couple miscellaneous fixes found in prolonged testing - Githubissues

redpanda-data / redpanda

Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM!

https://redpanda.com

9.72k stars 591 forks source link

iceberg: couple miscellaneous fixes found in prolonged testing #24225

Closed andrwng closed 1 week ago

andrwng commented 1 week ago

Addresses a couple issues seen when running the Iceberg integration for a moderate length of time:

oversized allocs from loading table metadata as a string
frequent EOSs that wouldn't be retried, and so we'd get stuck when performing many downloads in a row as a part of a merge append

Backports Required

[ ] none - not a bug fix
[ ] none - this is a backport
[ ] none - issue does not exist in previous branches
[ ] none - papercut/not impactful enough to backport
[x] v24.3.x
[ ] v24.2.x
[ ] v24.1.x

Release Notes

none

ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/58442#01934e48-5d39-4819-853d-9a517a87b6cc ducktape was retried in https://buildkite.com/redpanda/redpanda/builds/58442#01934e4c-c284-4f6a-a4c9-a2e8a9716289

vbotbuildovich commented 1 week ago

/backport v24.3.x