Incomplete Feature Set in Proxy Implementation Affecting S3 Server Mount Compatibility

[rG2ee]

It appears that the proxy implementation lacks some features to be considered complete. My goal is to implement an S3 server that can be mounted into the file systems of various programs.

The programs I have tested this with are Mountain Duck (https://mountainduck.io/) and ExpanDrive (https://www.expandrive.com/).

Initially, I integrated a MinIO server, for which example credentials can be found here: https://min.io/docs/minio/container/administration/minio-console.html#logging-in. The setup worked for both programs. However, upon attempting to integrate the S3S-proxy, I encountered issues: Mountain Duck displayed the buckets without any content, and ExpanDrive did not show anything at all (on macOS).

I have debugged the requests and it seems that in both cases, the list_objects function is not even being called, indicating some sort of signature problems.

I am unsure how to proceed or whether this issue is already known. Any guidance to further diagnose this problem would be greatly appreciated, as my experience with the S3 protocol is somewhat limited.

[Nugine]

Could you provide your detailed steps and debug logs?

[rG2ee]

Sure. I will provide them tomorrow.

Edit: To be clear. You mean the logs, when running with RUST_LOG=debug cargo run, right?

[Nugine]

See the example script. It enables debug logs by default.

https://github.com/Nugine/s3s/blob/ba5ae877a12f041c6b198b8c687a026ff007f499/scripts/s3s-proxy.sh#L16-L18

[rG2ee]

Here is an overview over what I have tried. Connecting directly via ExpanDrive is ok but using the proxy yields some errors.

Set up server and test data

mkdir -p /tmp/minio
docker run -p 9000:9000 -p 9001:9001 \
-e "MINIO_DOMAIN=localhost:9000" \
-e "MINIO_HTTP_TRACE=1" \
-v /tmp/minio:/data \
minio/minio:latest server /data --console-address ":9001" &

## install minio cli (https://min.io/docs/minio/linux/reference/minio-mc.html#install-mc)

## set up some test data
mc alias set local-minio http://localhost:9000 minioadmin minioadmin
mc mb local-minio/tmp
mc mb local-minio/tmp2
echo "Hello World" > /tmp/example.txt && mc cp /tmp/example.txt local-minio/tmp/example.txt

Connect direcly with Expandrive

(You can just enter user, user@123.de to start a free trial. No real email required.)

choose aws and enter credentials:

![2-enter-credentials](https://github.com/Nugine/s3s/assets/72554067/455e9a21-4f85-4a58-a2b

expected output is ok

f-24cd92c75a7d)

s3s-proxy

start server

export RUST_LOG="s3s_proxy=debug,s3s_aws=debug,s3s=debug"
export AWS_ACCESS_KEY_ID=minioadmin
export AWS_SECRET_ACCESS_KEY=minioadmin
export AWS_REGION=us-east-1
./target/debug/s3s-proxy \
    --host          localhost               \
    --port          8014                    \
    --domain-name   localhost:8014          \
    --endpoint-url  http://localhost:9000

connect via ExpanDrive

credentials

output

logs

s3s-proxy.logs.zip

[Nugine]

It seems that there is something wrong with signature v2 check in s3s.

[Nugine]

I have reproduced the bug successfully. The offical documentation of signature v2 has some conflict statements which are misleading. I'm digging into aws-sdk-js v2 source code to find the cause.

[Nugine]

It should work now. The changes in s3s will be included in v0.10.0 release, possibly in one month. You can also use git dependencies to test latest code.