Unclear to include trailing slash / in endpoint URL

scohajo commented 3 weeks ago

A note for the community

Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
If you are interested in working on this issue or have submitted a pull request, please leave a comment

Problem

In the docs, it's unclear that you need a trailing slash in the endpoint URL to build the correct bucket locator. I didn't discover why my instance was failing to find the bucket until I turned on trace logging and noticed the bucket URL was endpoint.combucketname instead of endpoint.com/bucketname. It would be much clearer if the example in the doc included the trailing slash and would be nice if the docs specified the slash as well.

Configuration

vector:
  sinks:
    bucketname:
      type: aws_s3
      bucket: "bucketname"
      region: nonstandard-region
      endpoint: https://endpoint.com
      key_prefix: "%F/{{kubernetes.pod_namespace}}/{{kubernetes.pod_name}}/"
      auth:
        access_key_id: ${AWS_ACCESS_KEY_ID}
        secret_access_key: ${AWS_SECRET_ACCESS_KEY}
      inputs:
        - in.kube-pods
      encoding: 
        codec: json
      compression: gzip
      tls:
        ca_file: "/path/to/ca.pem"
        crt_file: "/path/to/tls.crt"
        key_file: "/path/to/tls.key"

Version

0.33.0001

Debug Output

No response

Example Data

No response

Additional Context

No response

References

No response

scohajo commented 3 weeks ago

https://github.com/vectordotdev/vector/pull/20774

jszwedko commented 2 weeks ago

Reopening since I had to revert https://github.com/vectordotdev/vector/pull/20774 in https://github.com/vectordotdev/vector/pull/20791

jszwedko commented 2 weeks ago

@scohajo I actually took a closer look at this and think that I might be missing something. All of the examples that AWS has for configuring the endpoint for a service don't have a / (or a path at all for that matter): https://docs.aws.amazon.com/cli/v1/userguide/cli-configure-endpoints.html

Are you sure that the / is required? I'm wondering if this also might not be a bug in the AWS SDK: that it should be adding the trailing / if missing.

vectordotdev / vector