soto-project / soto

Swift SDK for AWS that works on Linux, macOS and iOS
https://soto.codes
Apache License 2.0
878 stars 83 forks source link

7.x Multipart upload not working - XML is wrong #735

Open Samsv77 opened 3 days ago

Samsv77 commented 3 days ago

Describe the bug Multipart upload is not working in 7.x, and fails with an XML error

After investigation, SOTO is sending the following XML content during Complete Multipart upload request <?xml version="1.0" encoding="UTF-8"?><MultipartUpload xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Part><ETag>"..."</ETag><PartNumber>1</PartNumber></Part></MultipartUpload>

This is wrong, the correct XML tag is CompleteMultipartUpload instead of MultipartUpload

Workaround: Implement a middleware to catch the XML and correct it

struct FixMultipartXMLMiddleware: AWSMiddlewareProtocol {
    func handle(
        _ request: SotoCore.AWSHTTPRequest, context: SotoCore.AWSMiddlewareContext,
        next: (SotoCore.AWSHTTPRequest, SotoCore.AWSMiddlewareContext) async throws -> SotoCore.AWSHTTPResponse
    ) async throws -> SotoCore.AWSHTTPResponse {
        var copy = request

        // Fix Soto 7 multipart upload XML request
        var content = String(buffer: try await copy.body.collect(upTo: .max))
        if content.starts(with: "<?xml") {
            content.replace("MultipartUpload", with: "CompleteMultipartUpload")
            copy.body = .init(string: content)
        }

        return try await next(copy, context)
    }
} 
adam-fowler commented 3 days ago

So I had a quick look at this and yes you are correct the top level tag should be CompleteMultipartUpload. And I've found out where the issue is. The code generator is ignoring the XmlName trait in the service model in this one particular case.

Having said that though, all my tests show multipart upload still working correctly ie the tag is ignored by AWS S3. Are you using an S3 like provider or actual AWS.

I will fix this, but am away on holidav just now.

Samsv77 commented 2 days ago

Yes I am using a private S3 provider. Thanks for having a look

Enjoy your holiday!