I get this error when using Beam with Cloudflare R2 S3 API. There is seemingly no way to tell Beam to use consistent part sizes.
apache_beam.io.aws.clients.s3.messages.S3ClientError: ("An error occurred (InvalidPart) when calling the CompleteMultipartUpload operation: All non-trailing parts must have the same length. [while running 'WriteSplit[train]/Write/Write/WriteImpl/WriteBundles']", 400)
Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
What happened?
https://developers.cloudflare.com/r2/objects/multipart-objects/ "All parts except the last one must be the same size. The last part has no minimum size, but must be the same or smaller than the other parts."
I get this error when using Beam with Cloudflare R2 S3 API. There is seemingly no way to tell Beam to use consistent part sizes.
Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
Issue Components