Large file upload fails

[emergentcomplex]

Describe the bug

I'm attempting to use Digital Ocean Spaces (ceph-backed storage). It's supposed to be S3 compatible. I can put objects smaller than ~2MB. Any objects over that size will error out.

Expected behavior

I expect object under 5GB to successfully upload to spaces using this sdk and give a 201 response.

Current behavior

500 response code error. The actual output message is:

Exception in thread "main" aws.sdk.kotlin.services.s3.model.S3Exception: The request signature we calculated does not match the signature you provided. Check your key and signing method.

Steps to Reproduce

package com.altechmechanical

import aws.sdk.kotlin.runtime.auth.credentials.StaticCredentialsProvider
import aws.sdk.kotlin.services.s3.S3Client
import aws.sdk.kotlin.services.s3.model.PutObjectRequest
import aws.smithy.kotlin.runtime.auth.awscredentials.CredentialsProvider
import aws.smithy.kotlin.runtime.content.asByteStream
import aws.smithy.kotlin.runtime.net.Host
import aws.smithy.kotlin.runtime.net.Scheme
import aws.smithy.kotlin.runtime.net.url.Url
import java.io.File
import java.net.URL

//TIP To <b>Run</b> code, press <shortcut actionId="Run"/> or
// click the <icon src="AllIcons.Actions.Execute"/> icon in the gutter.
suspend fun main() {
    suspend fun putS3Object(
        bucketName: String,
        objectKey: String,
        objectPath: String,
    ) {
        val metadataVal = mutableMapOf<String, String>()
        metadataVal["myVal"] = "test"

        val request =
            PutObjectRequest {
                bucket = bucketName
                key = objectKey
                metadata = metadataVal
                body = File(objectPath).asByteStream()
            }

        data class S3ClientConfig(
            val bucketName: String,
            val region: String,
            val url: URL,
            val credentials: CredentialsProvider,
        )

        val s3ClientConfig = S3ClientConfig(
            url = URL("https://sfo2.digitaloceanspaces.com"),
            bucketName = "altecmechanical",
            region = "sf02",
            credentials = StaticCredentialsProvider {
                accessKeyId = "XXXXXXXXXXXXXXXXX"
                secretAccessKey = "XXXXXXXXXXXXXXXXXX/tS+XXXXXXXXXXXX"
            },
        )

        S3Client {
            endpointUrl =
                Url {
                    scheme = Scheme.parse(s3ClientConfig.url.protocol)
                    host = Host.parse(s3ClientConfig.url.host)
                }
            region = s3ClientConfig.region
            credentialsProvider = s3ClientConfig.credentials
        }.use { s3 ->
            val response = s3.putObject(request)
            println("Tag information is ${response.eTag}")
        }
    }
    putS3Object("altecmechanical", "test.jpg", "test.jpg")
}

I ran this file in IntelliJ. It's a simplified version of what was failing in my KMM project. Same failure in this code. Will work with boto3 python aws sdk to Spaces with large file -- verified.

Possible Solution

It seems something is off with how the signature is calculated. I noticed there is a v2 and v4 out in the wild. There was mentioning of a shifting protocols with amazon. Perhaps this needs to be handled differently for DO Spaces?

Context

OpenJDK Runtime Environment (build 17.0.11+9-Ubuntu-1) IntelliJ IDEA Ultimate 2024.1.4

AWS SDK for Kotlin version

implementation("aws.sdk.kotlin:s3:1.2.38")}

Platform (JVM/JS/Native)

kotlin("jvm") version "1.9.0"

Operating system and version

Ubuntu 24.04LTS

[emergentcomplex]

Here's the section on spaces https://docs.digitalocean.com/reference/api/spaces-api/#authentication which goes over signatures. It claims both v4 and v2 are supported.

[emergentcomplex]

I've confirmed large files work OK with the latest AWS SDK for Java. Works with Python and Java with Spaces. Won't work with Kotlin :man_shrugging:

[lauzadis]

AWS SDK for Kotlin uses SigV4 for signing, so I don't think that's an issue.

We enable aws-chunked encoding by default for large S3 requests. It's possible Digital Ocean Spaces doesn't support that, so can you try disabling it by setting enableAwsChunked = false on the S3Client? Let me know if that fixes your issue.

[emergentcomplex]

Confirming

S3Client {
            enableAwsChunked = false

Did the trick :rainbow: thank you

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.