oxyno-zeta / s3-proxy

S3 Reverse Proxy with GET, PUT and DELETE methods and authentication (OpenID Connect and Basic Auth)
https://oxyno-zeta.github.io/s3-proxy/
Apache License 2.0
284 stars 31 forks source link

help in configuration #444

Closed nohant closed 3 months ago

nohant commented 3 months ago

Hello!

Im testing this project because i would like to access some files from a enclosed bucket. i already have configured a IAM profile that can access bucket and files inside it:

serviceAccount:
  annotations:
      eks.amazonaws.com/role-arn: ARNOFTHEROLE

but, if i configure the bucket this way:

targets:
  first-bucket:
    bucket:
        name: nameofthebucket
        region: regionofthebucket
        disableSSL: true
    mount:
      path:
        - /
    resources:
        - path: /*
          methods:
            - GET
    actions:
        GET:
          enabled: true
          config:
            disableListing: false

when i go to the ingress url, i only get as response a joyful: Not Found / i would like to be able to access the listing of the folder (i know there are folders inside the bucket.).

am i doing something wrong? Creds are needed outside of the IAM profile inside the annotation that sets the env? Thanks!

oxyno-zeta commented 3 months ago

Hello,

I think you can remove the resource block in your configuration as you don't have any authentication declared.

I don't use any service account IAM solution like kube2iam or anything like this so... I can't say it is well configured sorry.

Can you provide me the debug logs please? Maybe there another thing.

Best regards,

nohant commented 3 months ago

By using the service account it should use the Aws_session_token. (teoretically... xD ) moving to

log:
  # Log level
  level: debug

didnt changed anything at startup, but i get an error when testing the url from the ingress. do you need the really big stack? or do you think is needed?

oxyno-zeta commented 3 months ago

Can you provide me the configmap content please ? And the full helm value used to deploy it please ? I suspect something related to helm values.

In parallel, are you sure that the not found isn't presented by your ingress controller ?

nohant commented 3 months ago

here is the configmap:

apiVersion: v1
data:
  config.yaml: |-
    log:
      format: text
      level: info
    targets: {}
kind: ConfigMap
metadata:
  annotations:
    meta.helm.sh/release-name: aws-s3-proxy
    meta.helm.sh/release-namespace: support
  creationTimestamp: "2024-04-10T15:12:00Z"
  labels:
    app.kubernetes.io/instance: aws-s3-proxy
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: s3-proxy
    helm.sh/chart: s3-proxy-2.19.0
  name: aws-s3-proxy-config
  namespace: support
  resourceVersion: "484966012"
  uid: 28f8b0fc-239f-4fd5-840c-68ea80db7b91

and here the full values:

log:
  level: debug
  format: text

serviceAccount:
  annotations:
      eks.amazonaws.com/role-arn: MYSOPERARNROLE

ingress:
  enabled: true
  annotations:
    ingressClassName: internal
  hosts:
    - host: MYSUPERADDRESS
      paths:
        - path: /
          pathType: ImplementationSpecific

targets:
  first-bucket:
    bucket:
        name: MYSUPERBUCKET
        region: MYSUPERREGION
    mount:
      path:
        - /
    actions:
        GET:
          enabled: true
          config:
            disableListing: false

Thanks for your help! and no, thats not coming from my ingress nor my ingress controller.

oxyno-zeta commented 3 months ago

Ok that's why. The configmap is empty as you can see. Helm values aren't at the good level.

Here is the good ones:


serviceAccount:
  annotations:
      eks.amazonaws.com/role-arn: MYSOPERARNROLE

ingress:
  enabled: true
  annotations:
    ingressClassName: internal
  hosts:
    - host: MYSUPERADDRESS
      paths:
        - path: /
          pathType: ImplementationSpecific
#
# See configuration of project: https://oxyno-zeta.github.io/s3-proxy/
#
# Note: This can put as blank. The application can start without any configuration.
configFiles:
  config.yaml:
    targets:
      first-bucket:
       bucket:
         name: MYSUPERBUCKET
         region: MYSUPERREGION
       mount:
         path:
           - /
       actions:
        GET:
          enabled: true
          config:
            disableListing: false

All application configuration must be under the config files part of the values. See here: https://github.com/oxyno-zeta/helm-charts-v2/blob/master/charts/s3-proxy/values.yaml#L182

Have a good rest of your day !

nohant commented 3 months ago

Oh, now the error make sense.

connect: connection refused

this is problably due to the IAM question right? its not using correctly the serviceaccount and i have to generate a secret key to make this work.

nohant commented 3 months ago

ok. i can confirm it works with serviceaccount and iam permission after some other testing. the important part on my config was that i disabled SSL.

if you want i could (in the near future) open a PR for the documentation on how to use that and why. Thanks for the help, for real!