fluent / fluent-plugin-s3

Amazon S3 input and output plugin for Fluentd
https://docs.fluentd.org/output/s3
314 stars 217 forks source link

Placeholder replacement order #394

Closed tgilman closed 1 year ago

tgilman commented 2 years ago

Describe the bug

Unable to use placeholder values within interpolation expression for path property.

Example: path "#{'eye.test'.sub('.', '/') + '${tag}'.sub('.', '/')}/%Y-%m-%d/" ${tag} = 'his.blahbalh'

results in 'eye/testhis.blahblah' because the ${tag} placeholder replacement doesn't happen until after the outer #{} interpolation has already executed.

To Reproduce

See path value above.

Expected behavior

Shouldn't placeholder replacement happen prior to evaluating the resulting string?

Your Environment

- Fluentd version:
- TD Agent version:
- fluent-plugin-s3 version:
- aws-sdk-s3 version:
- aws-sdk-sqs version:
- Operating system:
- Kernel version:

Your Configuration

  <store>
    @type s3 #send logs to s3 for longterm storage
    s3_bucket fluend-poc
    s3_region us-west-2
    path "#{'eye.test'.sub('.', '/') + '${tag}'.sub('.', '/')}/%Y-%m-%d/"
    <buffer tag,time>
    @type file
    path /var/log/fluent/s3
    timekey 60 #900 # 15 min partition
    timekey_wait 1m
    timekey_use_utc true # use utc
    chunk_limit_size 256m
  </buffer>
  </store>

Your Error Log

No errors produced

Additional context

No response

github-actions[bot] commented 2 years ago

This issue has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days

github-actions[bot] commented 2 years ago

This issue has been automatically marked as stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days

github-actions[bot] commented 1 year ago

This issue was automatically closed because of stale in 30 days