search

bruce-szalwinski-he / aws-msk-iam-sasl-signer-ruby

MIT License
0 stars 0 forks source link

Seg Fault running in docker on Mac M2 #13

Closed kcculhwch closed 5 months ago

kcculhwch commented 5 months ago

I'm running with:

Then mounting my ~/.aws folder as a docker volume and then inside docker running IRB

irb(main):001:0> require "aws_msk_iam_sasl_signer"
=> true
irb(main):002:0> require "json"
irb(main):003:0> require "rdkafka"
=> true
irb(main):004:0> KAFKA_TOPIC = ENV['KAFKA_TOPIC'] || ""
=> "ingest"
irb(main):005:0> KAFKA_BOOTSTRAP_SERVERS = ENV['KAFKA_BOOTSTRAP_SERVERS'] || ""
=> "REDACTED.amazonaws.com:9098"
irb(main):006:0> KAFKA_ROLE = ENV['KAFKA_ROLE'] || ""
irb(main):007:0> KAFKA_REGION = ENV['KAFKA_REGION'] || ""
=> "us-east-2"
irb(main):008:1* kafka = {
irb(main):009:1*     "bootstrap.servers": KAFKA_BOOTSTRAP_SERVERS,
irb(main):010:1*     "security.protocol": 'sasl_ssl',
irb(main):011:1*     "sasl.mechanisms": 'OAUTHBEARER',
irb(main):012:1*     "client.id": 'ruby-producer',
irb(main):013:0> }
=> {:"bootstrap.servers"=>"REDACTED.amazonaws.com:9098", :"security.protocol"=>"sasl_ssl", :"sasl.mechanisms"=>"OAUTHBEARER", :"client.id"=>"ruby-producer"}
irb(main):014:1*     def kafka_refresh_token(client, config)
irb(main):015:1*       signer = AwsMskIamSaslSigner::MSKTokenProvider.new(region: KAFKA_REGION)
irb(main):016:1*       auth_token = signer.generate_auth_token #= signer.generate_auth_token_from_role_arn(
irb(main):017:1* #        KAFKA_ROLE
irb(main):018:1* #      )
irb(main):019:1*
irb(main):020:1*       error_buffer = FFI::MemoryPointer.from_string(' ' * 256)
irb(main):021:2*       response = Rdkafka::Bindings.rd_kafka_oauthbearer_set_token(
irb(main):022:2*         client, auth_token.token, auth_token.expiration_time_ms, 'kafka-cluster', nil, 0, error_buffer, 256
irb(main):023:1*       )
irb(main):024:1*       return unless response != 0
irb(main):025:1*
irb(main):026:2*       Rdkafka::Bindings.rd_kafka_oauthbearer_set_token_failure(client,
irb(main):027:1*                                                                "Failed to set token: #{error_buffer.read_string}")
irb(main):028:1*
irb(main):029:0>     end
=> :kafka_refresh_token
irb(main):030:0> Rdkafka::Config.oauthbearer_token_refresh_callback = method(:kafka_refresh_token)
=> #<Method: Object#kafka_refresh_token(client, config) (irb):14>
irb(main):031:0>  producer = Rdkafka::Config.new(kafka).producer
(irb):21: [BUG] Segmentation fault at 0x0000000000001128
ruby 3.2.4 (2024-04-23 revision af471c0e01) [aarch64-linux]

-- Control frame information -----------------------------------------------
c:0050 p:---- s:0292 e:000291 CFUNC  :rd_kafka_oauthbearer_set_token
c:0049 p:0048 s:0280 e:000279 METHOD (irb):21 [FINISH]
c:0048 p:---- s:0270 e:000269 CFUNC  :call
c:0047 p:0021 s:0264 e:000263 BLOCK  /usr/local/bundle/bundler/gems/rdkafka-ruby-4eb360f32cd3/lib/rdkafka/bindings.rb:238 [FINISH]
c:0046 p:---- s:0258 e:000257 CFUNC  :rd_kafka_poll
c:0045 p:0023 s:0252 e:000251 BLOCK  /usr/local/bundle/bundler/gems/rdkafka-ruby-4eb360f32cd3/lib/rdkafka/native_kafka.rb:45
c:0044 p:0044 s:0249 e:000248 METHOD /usr/local/bundle/bundler/gems/rdkafka-ruby-4eb360f32cd3/lib/rdkafka/native_kafka.rb:77
c:0043 p:0029 s:0245 e:000244 BLOCK  /usr/local/bundle/bundler/gems/rdkafka-ruby-4eb360f32cd3/lib/rdkafka/native_kafka.rb:90 [FINISH]
c:0042 p:---- s:0242 e:000241 CFUNC  :synchronize
c:0041 p:0006 s:0238 e:000237 METHOD /usr/local/bundle/bundler/gems/rdkafka-ruby-4eb360f32cd3/lib/rdkafka/native_kafka.rb:83
c:0040 p:0004 s:0233 e:000232 METHOD /usr/local/bundle/bundler/gems/rdkafka-ruby-4eb360f32cd3/lib/rdkafka/native_kafka.rb:39
c:0039 p:0047 s:0229 e:000228 METHOD /usr/local/bundle/bundler/gems/rdkafka-ruby-4eb360f32cd3/lib/rdkafka/native_kafka.rb:33 [FINISH]
c:0038 p:---- s:0220 e:000219 CFUNC  :new
c:0037 p:0061 s:0214 e:000212 METHOD /usr/local/bundle/bundler/gems/rdkafka-ruby-4eb360f32cd3/lib/rdkafka/config.rb:251
c:0036 p:0008 s:0203 E:000d18 EVAL   (irb):31 [FINISH]
c:0035 p:---- s:0199 e:000198 CFUNC  :eval
c:0034 p:0020 s:0191 e:000190 METHOD /usr/local/lib/ruby/3.2.0/irb/workspace.rb:119
c:0033 p:0153 s:0183 e:000181 METHOD /usr/local/lib/ruby/3.2.0/irb/context.rb:502
c:0032 p:0119 s:0170 e:000169 BLOCK  /usr/local/lib/ruby/3.2.0/irb.rb:588
c:0031 p:0024 s:0164 e:000163 METHOD /usr/local/lib/ruby/3.2.0/irb.rb:777
c:0030 p:0007 s:0158 e:000157 BLOCK  /usr/local/lib/ruby/3.2.0/irb.rb:567
c:0029 p:0126 s:0153 e:000152 BLOCK  /usr/local/lib/ruby/3.2.0/irb/ruby-lex.rb:267 [FINISH]
c:0028 p:---- s:0149 e:000148 CFUNC  :loop
c:0027 p:0005 s:0145 e:000144 BLOCK  /usr/local/lib/ruby/3.2.0/irb/ruby-lex.rb:249 [FINISH]
c:0026 p:---- s:0142 e:000141 CFUNC  :catch
c:0025 p:0010 s:0137 e:000136 METHOD /usr/local/lib/ruby/3.2.0/irb/ruby-lex.rb:248
c:0024 p:0050 s:0132 E:002428 METHOD /usr/local/lib/ruby/3.2.0/irb.rb:566
c:0023 p:0003 s:0127 e:000126 BLOCK  /usr/local/lib/ruby/3.2.0/irb.rb:500 [FINISH]
c:0022 p:---- s:0124 e:000123 CFUNC  :catch
c:0021 p:0050 s:0119 E:0024f0 METHOD /usr/local/lib/ruby/3.2.0/irb.rb:499
c:0020 p:0069 s:0113 e:000112 METHOD /usr/local/lib/ruby/3.2.0/irb.rb:421
c:0019 p:0012 s:0107 e:000106 TOP    /usr/local/lib/ruby/gems/3.2.0/gems/irb-1.6.2/exe/irb:11 [FINISH]
c:0018 p:---- s:0104 e:000103 CFUNC  :load
c:0017 p:0078 s:0099 e:000098 TOP    /usr/local/bin/irb:25 [FINISH]
c:0016 p:---- s:0094 e:000093 CFUNC  :load
c:0015 p:0064 s:0089 e:000088 METHOD /usr/local/bundle/gems/bundler-2.3.26/lib/bundler/cli/exec.rb:58
c:0014 p:0050 s:0083 e:000082 METHOD /usr/local/bundle/gems/bundler-2.3.26/lib/bundler/cli/exec.rb:23
c:0013 p:0034 s:0078 e:000077 METHOD /usr/local/bundle/gems/bundler-2.3.26/lib/bundler/cli.rb:486
c:0012 p:0054 s:0073 e:000072 METHOD /usr/local/bundle/gems/bundler-2.3.26/lib/bundler/vendor/thor/lib/thor/command.rb:27
c:0011 p:0040 s:0065 e:000064 METHOD /usr/local/bundle/gems/bundler-2.3.26/lib/bundler/vendor/thor/lib/thor/invocation.rb:127
c:0010 p:0213 s:0058 e:000057 METHOD /usr/local/bundle/gems/bundler-2.3.26/lib/bundler/vendor/thor/lib/thor.rb:392
c:0009 p:0008 s:0045 e:000044 METHOD /usr/local/bundle/gems/bundler-2.3.26/lib/bundler/cli.rb:31
c:0008 p:0044 s:0040 e:000039 METHOD /usr/local/bundle/gems/bundler-2.3.26/lib/bundler/vendor/thor/lib/thor/base.rb:485
c:0007 p:0008 s:0033 e:000032 METHOD /usr/local/bundle/gems/bundler-2.3.26/lib/bundler/cli.rb:25
c:0006 p:0058 s:0028 e:000027 BLOCK  /usr/local/bundle/gems/bundler-2.3.26/exe/bundle:48
c:0005 p:0007 s:0022 e:000021 METHOD /usr/local/bundle/gems/bundler-2.3.26/lib/bundler/friendly_errors.rb:120
c:0004 p:0147 s:0017 E:0021d8 TOP    /usr/local/bundle/gems/bundler-2.3.26/exe/bundle:36 [FINISH]
c:0003 p:---- s:0013 e:000012 CFUNC  :load
c:0002 p:0081 s:0008 E:001670 EVAL   /usr/local/bundle/bin/bundle:25 [FINISH]
c:0001 p:0000 s:0003 E:000300 DUMMY  [FINISH]

-- Ruby level backtrace information ----------------------------------------
/usr/local/bundle/bin/bundle:25:in `<main>'
/usr/local/bundle/bin/bundle:25:in `load'
/usr/local/bundle/gems/bundler-2.3.26/exe/bundle:36:in `<top (required)>'
/usr/local/bundle/gems/bundler-2.3.26/lib/bundler/friendly_errors.rb:120:in `with_friendly_errors'
/usr/local/bundle/gems/bundler-2.3.26/exe/bundle:48:in `block in <top (required)>'
/usr/local/bundle/gems/bundler-2.3.26/lib/bundler/cli.rb:25:in `start'
/usr/local/bundle/gems/bundler-2.3.26/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
/usr/local/bundle/gems/bundler-2.3.26/lib/bundler/cli.rb:31:in `dispatch'
/usr/local/bundle/gems/bundler-2.3.26/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
/usr/local/bundle/gems/bundler-2.3.26/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/usr/local/bundle/gems/bundler-2.3.26/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/usr/local/bundle/gems/bundler-2.3.26/lib/bundler/cli.rb:486:in `exec'
/usr/local/bundle/gems/bundler-2.3.26/lib/bundler/cli/exec.rb:23:in `run'
/usr/local/bundle/gems/bundler-2.3.26/lib/bundler/cli/exec.rb:58:in `kernel_load'
/usr/local/bundle/gems/bundler-2.3.26/lib/bundler/cli/exec.rb:58:in `load'
/usr/local/bin/irb:25:in `<top (required)>'
/usr/local/bin/irb:25:in `load'
/usr/local/lib/ruby/gems/3.2.0/gems/irb-1.6.2/exe/irb:11:in `<top (required)>'
/usr/local/lib/ruby/3.2.0/irb.rb:421:in `start'
/usr/local/lib/ruby/3.2.0/irb.rb:499:in `run'
/usr/local/lib/ruby/3.2.0/irb.rb:499:in `catch'
/usr/local/lib/ruby/3.2.0/irb.rb:500:in `block in run'
/usr/local/lib/ruby/3.2.0/irb.rb:566:in `eval_input'
/usr/local/lib/ruby/3.2.0/irb/ruby-lex.rb:248:in `each_top_level_statement'
/usr/local/lib/ruby/3.2.0/irb/ruby-lex.rb:248:in `catch'
/usr/local/lib/ruby/3.2.0/irb/ruby-lex.rb:249:in `block in each_top_level_statement'
/usr/local/lib/ruby/3.2.0/irb/ruby-lex.rb:249:in `loop'
/usr/local/lib/ruby/3.2.0/irb/ruby-lex.rb:267:in `block (2 levels) in each_top_level_statement'
/usr/local/lib/ruby/3.2.0/irb.rb:567:in `block in eval_input'
/usr/local/lib/ruby/3.2.0/irb.rb:777:in `signal_status'
/usr/local/lib/ruby/3.2.0/irb.rb:588:in `block (2 levels) in eval_input'
/usr/local/lib/ruby/3.2.0/irb/context.rb:502:in `evaluate'
/usr/local/lib/ruby/3.2.0/irb/workspace.rb:119:in `evaluate'
/usr/local/lib/ruby/3.2.0/irb/workspace.rb:119:in `eval'
(irb):31:in `<top (required)>'
/usr/local/bundle/bundler/gems/rdkafka-ruby-4eb360f32cd3/lib/rdkafka/config.rb:251:in `producer'
/usr/local/bundle/bundler/gems/rdkafka-ruby-4eb360f32cd3/lib/rdkafka/config.rb:251:in `new'
/usr/local/bundle/bundler/gems/rdkafka-ruby-4eb360f32cd3/lib/rdkafka/native_kafka.rb:33:in `initialize'
/usr/local/bundle/bundler/gems/rdkafka-ruby-4eb360f32cd3/lib/rdkafka/native_kafka.rb:39:in `start'
/usr/local/bundle/bundler/gems/rdkafka-ruby-4eb360f32cd3/lib/rdkafka/native_kafka.rb:83:in `synchronize'
/usr/local/bundle/bundler/gems/rdkafka-ruby-4eb360f32cd3/lib/rdkafka/native_kafka.rb:83:in `synchronize'
/usr/local/bundle/bundler/gems/rdkafka-ruby-4eb360f32cd3/lib/rdkafka/native_kafka.rb:90:in `block in synchronize'
/usr/local/bundle/bundler/gems/rdkafka-ruby-4eb360f32cd3/lib/rdkafka/native_kafka.rb:77:in `with_inner'
/usr/local/bundle/bundler/gems/rdkafka-ruby-4eb360f32cd3/lib/rdkafka/native_kafka.rb:45:in `block in start'
/usr/local/bundle/bundler/gems/rdkafka-ruby-4eb360f32cd3/lib/rdkafka/native_kafka.rb:45:in `rd_kafka_poll'
/usr/local/bundle/bundler/gems/rdkafka-ruby-4eb360f32cd3/lib/rdkafka/bindings.rb:238:in `block in <module:Bindings>'
/usr/local/bundle/bundler/gems/rdkafka-ruby-4eb360f32cd3/lib/rdkafka/bindings.rb:238:in `call'
(irb):21:in `kafka_refresh_token'
(irb):21:in `rd_kafka_oauthbearer_set_token'

(Truncated for length I can send the full memory map if you need)

Do I need an alternate version of librdkafka?

I have tried several variations and keep hitting the same error

bruce-szalwinski-he commented 5 months ago

The API to rdkafka changed and I haven't gotten back to updating the example. I'll knock out an updated example and get back with you.

bruce-szalwinski-he commented 5 months ago

See https://github.com/bruce-szalwinski-he/aws-msk-iam-sasl-signer-ruby/pull/14 for latest docs.

kcculhwch commented 5 months ago

Thank you for responding so quickly! I'll try it out on Monday.

On Sat, May 4, 2024 at 5:46 PM bruce-szalwinski-he @.***> wrote:

See #14 https://github.com/bruce-szalwinski-he/aws-msk-iam-sasl-signer-ruby/pull/14 for latest docs.

— Reply to this email directly, view it on GitHub https://github.com/bruce-szalwinski-he/aws-msk-iam-sasl-signer-ruby/issues/13#issuecomment-2094407870, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACVCIGFT2VGDAJ24YYJF3DLZAVJKVAVCNFSM6AAAAABHGFBSPGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJUGQYDOOBXGA . You are receiving this because you authored the thread.Message ID: <bruce-szalwinski-he/aws-msk-iam-sasl-signer-ruby/issues/13/2094407870@ github.com>