oxyno-zeta / s3-proxy

S3 Reverse Proxy with GET, PUT and DELETE methods and authentication (OpenID Connect and Basic Auth)
https://oxyno-zeta.github.io/s3-proxy/
Apache License 2.0
295 stars 33 forks source link

Allow application/octet-stream #325

Closed HappyFX closed 1 year ago

HappyFX commented 1 year ago

Is your feature request related to a problem? Please describe. I have configured S3 proxy to use from gitlab runner and they are uploading with Content-Type: application/octet-stream

Describe the solution you'd like Add support to PUT with Content-Type: application/octet-stream

Describe alternatives you've considered

Gitlab not support yet https://gitlab.com/gitlab-org/gitlab-runner/-/issues/26921

Additional context

dump of request:

PUT /gitlab-runner/cache/project/1/job-non_protected?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=1%2F20221118%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20221118T074923Z&X-Amz-Expires=3600&X-Amz-SignedHeaders=host&X-Amz-Signature=b6548e02e85c4f93463f0555238f04b85d9bf97db456cbc90edb825bc4b92520 HTTP/1.1
Host: s3-proxy.host:8080
User-Agent: Go-http-client/1.1
Content-Length: 237
Content-Type: application/octet-stream
Last-Modified: Fri, 18 Nov 2022 07:49:23 GMT

PK........+>rU..............!.cached_fileux.............UT....9wcUT....9wc*..MU..-./*I.+QHI,I.241312.4.......PK..$.!L#.......PK..........+>rU$.!L#.........!...............cached_fileux.............UT....9wcUT....9wcPK..........Z...}.....HTTP/1.1 500 Internal Server Error
Cache-Control: no-cache, no-store, no-transform, must-revalidate, private, max-age=0
Content-Type: text/html; charset=utf-8
Expires: Thu, 01 Jan 1970 00:00:00 UTC
Pragma: no-cache
X-Accel-Expires: 0
Date: Fri, 18 Nov 2022 07:49:23 GMT
Content-Length: 142

<!DOCTYPE html>
<html>
  <body>
    <h1>Internal Server Error</h1>
    <p>request Content-Type isn't multipart/form-data</p>
  </body>
</html>

Proxy config is

# Log configuration
log:
  # Log level
  level: debug
  # Log format
  format: text
  # Log file path
  # filePath:
targets:
  first-bucket:
    ## Mount point
    mount:
      path:
        - /gitlab-runner/
    actions:
      PUT:
        enabled: true
        allowOverride: true
    ## Bucket configuration
    bucket:
      name: gitlab-runner
      s3Endpoint: test.host:8080
      disableSSL: true
      credentials:
        accessKey:
          value: ***
        secretKey:
          value: ***
oxyno-zeta commented 1 year ago

Hello @HappyFX ,

The problem is happening because this project is a s3-proxy for serving content and not to proxy the full S3 API. I think you are looking for a project like this one: https://github.com/gaul/s3proxy

I'm closing this issue. Feel free to reopen it if you need it.

Regards,

Oxyno-zeta