search

atomita / fluent-plugin-aws-elasticsearch-service

This repository is extends of https://github.com/uken/fluent-plugin-elasticsearch/ which made connectable to Amazon Elassticsearch Service using Aws Signers V4. Please check the requirements to connect to Amazon Elassticsearch Service without using Aws Signers V4 for before starting use. (reference https://groups.google.com/forum/#!msg/fluentd/uW87VAOqxeE/cfhenicEBAAJ)
MIT License
122 stars 49 forks source link

changed the paremeter service_name to service for faraday. #82

Closed mvlbarcelos closed 2 years ago

mvlbarcelos commented 2 years ago

faraday_middleware-aws-sigv4 changed to use service instead of service_name https://github.com/winebarrel/faraday_middleware-aws-sigv4#upgrading-from-faraday_middleware-aws-signers-v4

require 'aws-sdk'
require 'elasticsearch'
require 'faraday_middleware/aws_sigv4'

region = 'eu-central-1'
service = 'es'

credentials = Aws::AssumeRoleWebIdentityCredentials.new({
    role_arn: 'arn:aws:iam::ACCOUNT:role/ROLE',
    web_identity_token_file: '/var/run/secrets/eks.amazonaws.com/serviceaccount/token',
    region: 'eu-central-1'
}).credentials

full_url_and_port = 'HOST'
index = 'logs-marcus-test'
type = '_doc'
id = '1'
document = {
  year: 2007,
  title: '5 Centimeters per Second',
  info: {
    plot: 'Told in three interconnected segments, we follow a young man named Takaki through his life.',
    rating: 7.7
  }
}

client = Elasticsearch::Client.new(url: full_url_and_port) do |f|
  f.request :aws_sigv4,
    service_name: service,
    region: region,
    credentials: credentials
end

puts client.index index: index, type: type, id: id, body: document
Traceback (most recent call last):
       16: from /usr/local/bundle/gems/elasticsearch-transport-7.13.3/lib/elasticsearch/transport/transport/base.rb:288:in `perform_request'
       15: from /usr/local/bundle/gems/elasticsearch-transport-7.13.3/lib/elasticsearch/transport/transport/http/faraday.rb:48:in `block in perform_request'
       14: from /usr/local/bundle/gems/faraday-1.10.0/lib/faraday/connection.rb:516:in `run_request'
       13: from /usr/local/bundle/gems/faraday-1.10.0/lib/faraday/rack_builder.rb:154:in `build_response'
       12: from /usr/local/bundle/gems/faraday-1.10.0/lib/faraday/rack_builder.rb:167:in `app'
       11: from /usr/local/bundle/gems/faraday-1.10.0/lib/faraday/rack_builder.rb:174:in `to_app'
       10: from /usr/local/bundle/gems/faraday-1.10.0/lib/faraday/rack_builder.rb:174:in `inject'
        9: from /usr/local/bundle/gems/faraday-1.10.0/lib/faraday/rack_builder.rb:174:in `each'
        8: from /usr/local/bundle/gems/faraday-1.10.0/lib/faraday/rack_builder.rb:175:in `block in to_app'
        7: from /usr/local/bundle/gems/faraday-1.10.0/lib/faraday/rack_builder.rb:57:in `build'
        6: from /usr/local/bundle/gems/faraday-1.10.0/lib/faraday/dependency_loader.rb:21:in `new'
        5: from /usr/local/bundle/gems/faraday-1.10.0/lib/faraday/dependency_loader.rb:21:in `new'
        4: from /usr/local/bundle/gems/faraday_middleware-aws-sigv4-0.3.0/lib/faraday_middleware/request/aws_sigv4.rb:9:in `initialize'
        3: from /usr/local/bundle/gems/faraday_middleware-aws-sigv4-0.3.0/lib/faraday_middleware/request/aws_sigv4.rb:9:in `new'
        2: from /usr/local/bundle/gems/aws-sigv4-1.4.0/lib/aws-sigv4/signer.rb:122:in `initialize'
        1: from /usr/local/bundle/gems/aws-sigv4-1.4.0/lib/aws-sigv4/signer.rb:609:in `extract_service'
ArgumentError (missing required option :service)
fluent@marcus-fluentd:/$ gem list

*** LOCAL GEMS ***

async (1.30.0)
async-http (0.54.0)
async-io (1.32.2)
async-pool (0.3.8)
aws-eventstream (1.2.0)
aws-partitions (1.566.0)
aws-sdk-core (3.130.0)
aws-sigv4 (1.4.0)
bigdecimal (default: 1.4.1)
bundler (default: 1.17.2)
cmath (default: 1.0.0)
concurrent-ruby (1.1.9)
console (1.13.1)
cool.io (1.7.1)
csv (default: 3.0.9)
date (default: 2.0.0)
dbm (default: 1.0.0)
did_you_mean (1.3.0)
e2mmap (default: 0.1.0)
elasticsearch (7.13.3)
elasticsearch-api (7.13.3)
elasticsearch-transport (7.13.3)
etc (default: 1.0.1)
excon (0.92.0)
ext_monitor (0.1.2)
faraday (1.10.0)
faraday-em_http (1.0.0)
faraday-em_synchrony (1.0.0)
faraday-excon (1.1.0)
faraday-httpclient (1.0.1)
faraday-multipart (1.0.3)
faraday-net_http (1.0.1)
faraday-net_http_persistent (1.2.0)
faraday-patron (1.0.0)
faraday-rack (1.0.0)
faraday-retry (1.0.3)
faraday_middleware-aws-sigv4 (0.3.0)
fcntl (default: 1.0.0)
fiber-local (1.0.0)
fiddle (default: 1.0.0)
fileutils (default: 1.1.0)
fluent-plugin-aws-elasticsearch-service (2.4.1)
fluent-plugin-dedot_filter (1.0.0)
fluent-plugin-elasticsearch (5.2.0)
fluent-plugin-prometheus (2.0.2)
fluent-plugin-redaction (0.1.2)
fluentd (1.13.3)
forwardable (default: 1.2.0)
gdbm (default: 2.0.0)
http_parser.rb (0.7.0)
io-console (default: 0.4.7)
ipaddr (default: 1.2.2)
irb (default: 1.0.0)
jmespath (1.6.1)
json (2.4.1, default: 2.1.0)
logger (default: 1.3.0)
matrix (default: 0.1.0)
minitest (5.11.3)
msgpack (1.4.2)
multi_json (1.15.0)
multipart-post (2.1.1)
mutex_m (default: 0.1.0)
net-telnet (0.2.0)
nio4r (2.5.7)
oj (3.10.18)
openssl (default: 2.1.2)
ostruct (default: 0.1.0)
power_assert (1.1.3)
prime (default: 0.1.0)
prometheus-client (3.0.0)
protocol-hpack (1.4.2)
protocol-http (0.21.0)
protocol-http1 (0.13.2)
protocol-http2 (0.14.2)
psych (default: 3.1.0)
rake (12.3.3)
rdoc (default: 6.1.2.1)
rexml (default: 3.1.9.1)
rss (default: 0.2.7)
ruby2_keywords (0.0.5)
scanf (default: 1.0.0)
sdbm (default: 1.0.0)
serverengine (2.2.4)
shell (default: 0.7)
sigdump (0.2.4)
stringio (default: 0.0.2)
strptime (0.2.5)
strscan (default: 1.0.0)
sync (default: 0.5.0)
test-unit (3.2.9)
thwait (default: 0.1.0)
timers (4.3.3)
tracer (default: 0.1.0)
tzinfo (2.0.4)
tzinfo-data (1.2021.1)
webrick (default: 1.4.4)
xmlrpc (0.3.0)
yajl-ruby (1.4.1)
zlib (default: 1.0.0)