search

logstash-plugins / logstash-input-google_cloud_storage

Apache License 2.0
4 stars 9 forks source link

plugin crashes on 7.4 #5

Closed kribor closed 5 years ago

kribor commented 5 years ago

So I went through upgrading to 7.4 today and most things went smoothly except logstash which chrashes with the below stacktrace.

Platform: Docker Image base: docker.elastic.co/logstash/logstash:7.4.0 Running custom image based on this with plugin installed "RUN /usr/share/logstash/bin/logstash-plugin install logstash-input-google_cloud_storage logstash-input-google_pubsub " (installation is fine) JVM: OpenJDK 64-Bit Server VM 18.9 (build 11.0.4+11-LTS, mixed mode, sharing)

warning: thread "[lb-access-logs-pipeline]-pipeline-manager" terminated with exception (report_on_exception is true):
java.lang.NoClassDefFoundError: Could not initialize class io.opencensus.contrib.http.util.CloudTraceFormat
    at io.opencensus.contrib.http.util.HttpPropagationUtil.getCloudTraceFormat(io/opencensus/contrib/http/util/HttpPropagationUtil.java:39)
    at com.google.cloud.http.CensusHttpModule.<init>(com/google/cloud/http/CensusHttpModule.java:125)
    at com.google.cloud.storage.spi.v1.HttpStorageRpc.<init>(com/google/cloud/storage/spi/v1/HttpStorageRpc.java:107)
    at com.google.cloud.storage.StorageOptions$DefaultStorageRpcFactory.create(com/google/cloud/storage/StorageOptions.java:54)
    at com.google.cloud.storage.StorageOptions$DefaultStorageRpcFactory.create(com/google/cloud/storage/StorageOptions.java:48)
    at com.google.cloud.ServiceOptions.getRpc(com/google/cloud/ServiceOptions.java:512)
    at com.google.cloud.storage.StorageOptions.getStorageRpcV1(com/google/cloud/storage/StorageOptions.java:120)
    at com.google.cloud.storage.StorageImpl.<init>(com/google/cloud/storage/StorageImpl.java:102)
    at com.google.cloud.storage.StorageOptions$DefaultStorageFactory.create(com/google/cloud/storage/StorageOptions.java:44)
    at com.google.cloud.storage.StorageOptions$DefaultStorageFactory.create(com/google/cloud/storage/StorageOptions.java:38)
    at com.google.cloud.ServiceOptions.getService(com/google/cloud/ServiceOptions.java:499)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(jdk/internal/reflect/NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(jdk/internal/reflect/DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:566)
    at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:440)
    at org.jruby.javasupport.JavaMethod.invokeDirect(org/jruby/javasupport/JavaMethod.java:304)
    at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_google_cloud_storage_minus_0_dot_11_dot_1_minus_java.lib.logstash.inputs.cloud_storage.client.initialize_storage(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-google_cloud_storage-0.11.1-java/lib/logstash/inputs/cloud_storage/client.rb:32)
    at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_google_cloud_storage_minus_0_dot_11_dot_1_minus_java.lib.logstash.inputs.cloud_storage.client.RUBY$method$initialize_storage$0$__VARARGS__(usr/share/logstash/vendor/bundle/jruby/$2_dot_5_dot_0/gems/logstash_minus_input_minus_google_cloud_storage_minus_0_dot_11_dot_1_minus_java/lib/logstash/inputs/cloud_storage//usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-google_cloud_storage-0.11.1-java/lib/logstash/inputs/cloud_storage/client.rb)
    at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_google_cloud_storage_minus_0_dot_11_dot_1_minus_java.lib.logstash.inputs.cloud_storage.client.initialize(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-google_cloud_storage-0.11.1-java/lib/logstash/inputs/cloud_storage/client.rb:18)
    at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_google_cloud_storage_minus_0_dot_11_dot_1_minus_java.lib.logstash.inputs.cloud_storage.client.RUBY$method$initialize$0$__VARARGS__(usr/share/logstash/vendor/bundle/jruby/$2_dot_5_dot_0/gems/logstash_minus_input_minus_google_cloud_storage_minus_0_dot_11_dot_1_minus_java/lib/logstash/inputs/cloud_storage//usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-google_cloud_storage-0.11.1-java/lib/logstash/inputs/cloud_storage/client.rb)
    at org.jruby.RubyClass.newInstance(org/jruby/RubyClass.java:915)
    at org.jruby.RubyClass$INVOKER$i$newInstance.call(org/jruby/RubyClass$INVOKER$i$newInstance.gen)
    at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_google_cloud_storage_minus_0_dot_11_dot_1_minus_java.lib.logstash.inputs.google_cloud_storage.register(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-google_cloud_storage-0.11.1-java/lib/logstash/inputs/google_cloud_storage.rb:42)
    at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_google_cloud_storage_minus_0_dot_11_dot_1_minus_java.lib.logstash.inputs.google_cloud_storage.RUBY$method$register$0$__VARARGS__(usr/share/logstash/vendor/bundle/jruby/$2_dot_5_dot_0/gems/logstash_minus_input_minus_google_cloud_storage_minus_0_dot_11_dot_1_minus_java/lib/logstash/inputs//usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-google_cloud_storage-0.11.1-java/lib/logstash/inputs/google_cloud_storage.rb)
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.register_plugins(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:195)
    at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1800)
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.register_plugins(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:194)
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$register_plugins$0$__VARARGS__(usr/share/logstash/logstash_minus_core/lib/logstash//usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb)
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_inputs(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:296)
    at RUBY.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:252)
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.run(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:149)
    at RUBY.start(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:108)
    at org.jruby.RubyProc.call(org/jruby/RubyProc.java:295)
    at org.jruby.RubyProc.call(org/jruby/RubyProc.java:274)
    at org.jruby.RubyProc.call(org/jruby/RubyProc.java:270)
    at java.lang.Thread.run(java/lang/Thread.java:834)
warning: thread "[gae-access-logs-pipeline]-pipeline-manager" terminated with exception (report_on_exception is true):
[2019-10-02T07:53:07,937][ERROR][logstash.agent           ] Failed to execute action {:id=>:"lb-access-logs-pipeline", :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<lb-access-logs-pipeline>, action_result: false", :backtrace=>nil}
[2019-10-02T07:53:07,946][ERROR][logstash.agent           ] Failed to execute action {:id=>:"gae-access-logs-pipeline", :action_type=>LogStash::ConvergeResult::FailedAction, :message=>"Could not execute action: PipelineAction::Create<gae-access-logs-pipeline>, action_result: false", :backtrace=>nil}
java.lang.NoClassDefFoundError: io/opencensus/trace/Tracestate
    at io.opencensus.contrib.http.util.CloudTraceFormat.<clinit>(io/opencensus/contrib/http/util/CloudTraceFormat.java:81)
    at io.opencensus.contrib.http.util.HttpPropagationUtil.getCloudTraceFormat(io/opencensus/contrib/http/util/HttpPropagationUtil.java:39)
    at com.google.cloud.http.CensusHttpModule.<init>(com/google/cloud/http/CensusHttpModule.java:125)
    at com.google.cloud.storage.spi.v1.HttpStorageRpc.<init>(com/google/cloud/storage/spi/v1/HttpStorageRpc.java:107)
    at com.google.cloud.storage.StorageOptions$DefaultStorageRpcFactory.create(com/google/cloud/storage/StorageOptions.java:54)
    at com.google.cloud.storage.StorageOptions$DefaultStorageRpcFactory.create(com/google/cloud/storage/StorageOptions.java:48)
    at com.google.cloud.ServiceOptions.getRpc(com/google/cloud/ServiceOptions.java:512)
    at com.google.cloud.storage.StorageOptions.getStorageRpcV1(com/google/cloud/storage/StorageOptions.java:120)
    at com.google.cloud.storage.StorageImpl.<init>(com/google/cloud/storage/StorageImpl.java:102)
    at com.google.cloud.storage.StorageOptions$DefaultStorageFactory.create(com/google/cloud/storage/StorageOptions.java:44)
    at com.google.cloud.storage.StorageOptions$DefaultStorageFactory.create(com/google/cloud/storage/StorageOptions.java:38)
    at com.google.cloud.ServiceOptions.getService(com/google/cloud/ServiceOptions.java:499)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(jdk/internal/reflect/NativeMethodAccessorImpl.java:62)
    at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(jdk/internal/reflect/DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:566)
    at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(org/jruby/javasupport/JavaMethod.java:425)
    at org.jruby.javasupport.JavaMethod.invokeDirect(org/jruby/javasupport/JavaMethod.java:292)
    at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_google_cloud_storage_minus_0_dot_11_dot_1_minus_java.lib.logstash.inputs.cloud_storage.client.initialize_storage(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-google_cloud_storage-0.11.1-java/lib/logstash/inputs/cloud_storage/client.rb:32)
    at RUBY.initialize(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-google_cloud_storage-0.11.1-java/lib/logstash/inputs/cloud_storage/client.rb:18)
    at org.jruby.RubyClass.newInstance(org/jruby/RubyClass.java:915)
    at org.jruby.RubyClass$INVOKER$i$newInstance.call(org/jruby/RubyClass$INVOKER$i$newInstance.gen)
    at RUBY.register(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-google_cloud_storage-0.11.1-java/lib/logstash/inputs/google_cloud_storage.rb:42)
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.register_plugins(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:195)
    at org.jruby.RubyArray.each(org/jruby/RubyArray.java:1800)
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.register_plugins(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:194)
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$register_plugins$0$__VARARGS__(usr/share/logstash/logstash_minus_core/lib/logstash//usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb)
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start_inputs(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:296)
    at RUBY.start_workers(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:252)
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.run(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:149)
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$run$0$__VARARGS__(usr/share/logstash/logstash_minus_core/lib/logstash//usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb)
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.start(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:108)
    at org.jruby.RubyProc.call(org/jruby/RubyProc.java:295)
    at org.jruby.RubyProc.call(org/jruby/RubyProc.java:274)
    at org.jruby.RubyProc.call(org/jruby/RubyProc.java:270)
    at java.lang.Thread.run(java/lang/Thread.java:834)
Caused by: java.lang.ClassNotFoundException: io.opencensus.trace.Tracestate
    at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:471)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:588)
    at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:521)
    at io.opencensus.contrib.http.util.CloudTraceFormat.<clinit>(CloudTraceFormat.java:81)
    at io.opencensus.contrib.http.util.HttpPropagationUtil.getCloudTraceFormat(HttpPropagationUtil.java:39)
    at com.google.cloud.http.CensusHttpModule.<init>(CensusHttpModule.java:125)
    at com.google.cloud.storage.spi.v1.HttpStorageRpc.<init>(HttpStorageRpc.java:107)
    at com.google.cloud.storage.StorageOptions$DefaultStorageRpcFactory.create(StorageOptions.java:54)
    at com.google.cloud.storage.StorageOptions$DefaultStorageRpcFactory.create(StorageOptions.java:48)
    at com.google.cloud.ServiceOptions.getRpc(ServiceOptions.java:512)
    at com.google.cloud.storage.StorageOptions.getStorageRpcV1(StorageOptions.java:120)
    at com.google.cloud.storage.StorageImpl.<init>(StorageImpl.java:102)
    at com.google.cloud.storage.StorageOptions$DefaultStorageFactory.create(StorageOptions.java:44)
    at com.google.cloud.storage.StorageOptions$DefaultStorageFactory.create(StorageOptions.java:38)
    at com.google.cloud.ServiceOptions.getService(ServiceOptions.java:499)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:425)
    at org.jruby.javasupport.JavaMethod.invokeDirect(JavaMethod.java:292)
    at org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:28)
    at org.jruby.java.invokers.InstanceMethodInvoker.call(InstanceMethodInvoker.java:90)
    at org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)
    at usr.share.logstash.vendor.bundle.jruby.$2_dot_5_dot_0.gems.logstash_minus_input_minus_google_cloud_storage_minus_0_dot_11_dot_1_minus_java.lib.logstash.inputs.cloud_storage.client.RUBY$method$initialize_storage$0(/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-google_cloud_storage-0.11.1-java/lib/logstash/inputs/cloud_storage/client.rb:32)
    at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:117)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:156)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:200)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:375)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:174)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:316)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:105)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:92)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:84)
    at org.jruby.RubyClass.newInstance(RubyClass.java:915)
    at org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:86)
    at org.jruby.ir.instructions.CallBase.interpret(CallBase.java:540)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:361)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:105)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:92)
    at org.jruby.ir.targets.InvokeSite.fail(InvokeSite.java:223)
    at org.jruby.ir.targets.InvokeSite.fail(InvokeSite.java:230)
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$block$register_plugins$1(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:195)
    at org.jruby.runtime.CompiledIRBlockBody.yieldDirect(CompiledIRBlockBody.java:146)
    at org.jruby.runtime.BlockBody.yield(BlockBody.java:114)
    at org.jruby.runtime.Block.yield(Block.java:170)
    at org.jruby.RubyArray.each(RubyArray.java:1800)
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$register_plugins$0(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:194)
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$register_plugins$0$__VARARGS__(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb)
    at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)
    at org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$start_inputs$0(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:296)
    at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:104)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:123)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:192)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:354)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:143)
    at org.jruby.ir.interpreter.InterpreterEngine.processCall(InterpreterEngine.java:345)
    at org.jruby.ir.interpreter.StartupInterpreterEngine.interpret(StartupInterpreterEngine.java:72)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.INTERPRET_METHOD(MixedModeIRMethod.java:105)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:92)
    at org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$run$0(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:149)
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$run$0$__VARARGS__(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb)
    at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)
    at org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)
    at org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)
    at usr.share.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$block$start$1(/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:108)
    at org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:136)
    at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:77)
    at org.jruby.runtime.Block.call(Block.java:129)
    at org.jruby.RubyProc.call(RubyProc.java:295)
    at org.jruby.RubyProc.call(RubyProc.java:274)
    at org.jruby.RubyProc.call(RubyProc.java:270)
    at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)
    at java.base/java.lang.Thread.run(Thread.java:834)
[2019-10-02T07:53:08,385][ERROR][org.logstash.Logstash    ] java.lang.NoClassDefFoundError: Could not initialize class io.opencensus.contrib.http.util.CloudTraceFormat
kribor commented 5 years ago

I tried working around this issue by adding jars for the missing classes:

opencensus-contrib-http-util-0.24.0.jar opencensus-api-0.24.0.jar grpc-context-1.24.0.jar

and it appears to have worked, however then my pubsub input crashed instead.

kribor commented 5 years ago

Not sure what changed since yesterday but this is no longer happens as of today.