search

elastic / logstash

Logstash - transport and process your logs, events, or other data
https://www.elastic.co/products/logstash
Other
14.18k stars 3.5k forks source link

logstash running on linux ppc64 crashes on startup. #2747

Closed ralphbellofatto closed 9 years ago

ralphbellofatto commented 9 years ago

When I run logstash on a linux ppc64 machine as follows:

 bin/logstash -e 'input { stdin { } } output { stdout {} }'

It crashes with

    LoadError: Could not load FFI Provider: (NotImplementedError) FFI not available: null
     See http://jira.codehaus.org/browse/JRUBY-4583
      require at org/jruby/RubyKernel.java:1085
      require at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55
       (root) at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/ffi/ffi.rb:69
      require at org/jruby/RubyKernel.java:1085
       (root) at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:1
      require at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55
      require at org/jruby/RubyKernel.java:1085
       (root) at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/ffi.rb:1
       (root) at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:1
      require at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55
      require at org/jruby/RubyKernel.java:1085
         LibC at /opt/logstash/lib/logstash/util/prctl.rb:4
       (root) at /opt/logstash/lib/logstash/util/prctl.rb:3
         main at /opt/logstash/lib/logstash/runner.rb:79
       (root) at /opt/logstash/lib/logstash/runner.rb:215

searching the web we find that the suggested solution is to point the java.io.tmpdir to something other than /tmp (the assumption being that /tmp was mounted noexec), as follows:

JAVA_OPTS="-Djava.io.tmpdir=/opt/tmp" bin/logstash -e 'input { stdin { } } output { stdout {} }'

doing that produced the exact same error.

We are running RHEL6 on a Power 7 ppc74 machine. Has log stash been demonstrated to work on this combination of distribution, operating system and architecture.

ph commented 9 years ago

All the information I gather seem related to the /tmp directory too, what permissions are set on /opt/tmp ?

colinsurprenant commented 9 years ago

No, this is related to FFI & PPC64 see https://logstash.jira.com/browse/LOGSTASH-2234 and https://github.com/jnr/jffi/issues/5

ralphbellofatto commented 9 years ago

777

[root@bgxcat logstash]# ls -l /opt/tmp total 4 -rwxr-xr-x 1 root root 20 Mar 2 15:04 x.sh

Ralph Bellofatto IBM TJ Watson Research 1-914-945-3321 ralphbel@us.ibm.com

From: Pier-Hugues Pellerin notifications@github.com To: elasticsearch/logstash logstash@noreply.github.com Cc: Ralph Bellofatto/Watson/IBM@IBMUS Date: 03/03/2015 09:10 AM Subject: Re: [logstash] logstash running on linux ppc64 crashes on startup. (#2747)

All the information I gather seem related to the /tmp directory too, what permissions are set on /opt/tmp ?

— Reply to this email directly or view it on GitHub.

colinsurprenant commented 9 years ago

@ralphbellofatto we will followup on this but in the meantime there is a workaround if you are not using the file input plugin by disabling FFI using:

JAVA_OPTS=-Djruby.native.enabled=false bin/logstash ...

let us know if this help

ralphbellofatto commented 9 years ago

this issue: jnr/jffi#5 refers to the ppc64el (little endian) archtiecture, we are running on ppc64 for now (eventually we will want to run on the ppc64el., but not today)...

Ralph Bellofatto IBM TJ Watson Research 1-914-945-3321 ralphbel@us.ibm.com

ralphbellofatto commented 9 years ago

I tried the solution suggested https://logstash.jira.com/browse/LOGSTASH-2234 and jnr/jffi#5 and got:

[root@bgxcat logstash]# JAVA_OPTS=-Djruby.native.enabled=false bin/logstash 'input { stdin { } } output { stdout {} }' LoadError: Native API access is disabled require at org/jruby/RubyKernel.java:1085 require at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55 (root) at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/ffi/ffi.rb:69 require at org/jruby/RubyKernel.java:1085 (root) at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:1 require at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55 require at org/jruby/RubyKernel.java:1085 (root) at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/ffi.rb:1 (root) at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:1 require at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55 require at org/jruby/RubyKernel.java:1085 LibC at /opt/logstash/lib/logstash/util/prctl.rb:4 (root) at /opt/logstash/lib/logstash/util/prctl.rb:3 main at /opt/logstash/lib/logstash/runner.rb:79 (root) at /opt/logstash/lib/logstash/runner.rb:215

Ralph Bellofatto IBM TJ Watson Research 1-914-945-3321 ralphbel@us.ibm.com

ralphbellofatto commented 9 years ago

I executed the following to try to find more about this:

JAVA_OPTS="-Djruby.native.verbose=true" bin/logstash -e 'input { stdin { } } output { stdout {} }

and got the following trace.

java.lang.UnsatisfiedLinkError: could not load FFI provider jnr.ffi.provider.jffi.Provider
    at jnr.ffi.provider.InvalidProvider$1.loadLibrary(InvalidProvider.java:30)
    at jnr.ffi.LibraryLoader.load(LibraryLoader.java:228)
    at jnr.ffi.Library.loadLibrary(Library.java:123)
    at jnr.posix.POSIXFactory$DefaultLibCProvider$SingletonHolder.<clinit>(POSIXFactory.java:219)
    at java.lang.J9VMInternals.initializeImpl(Native Method)
    at java.lang.J9VMInternals.initialize(J9VMInternals.java:236)
    at jnr.posix.POSIXFactory$DefaultLibCProvider.getLibC(POSIXFactory.java:223)
    at jnr.posix.BaseNativePOSIX.<init>(BaseNativePOSIX.java:37)
    at jnr.posix.LinuxPOSIX.<init>(LinuxPOSIX.java:20)
    at jnr.posix.POSIXFactory.loadLinuxPOSIX(POSIXFactory.java:95)
    at jnr.posix.POSIXFactory.loadNativePOSIX(POSIXFactory.java:69)
    at jnr.posix.POSIXFactory.loadPOSIX(POSIXFactory.java:38)
    at jnr.posix.LazyPOSIX.loadPOSIX(LazyPOSIX.java:33)
    at jnr.posix.LazyPOSIX.posix(LazyPOSIX.java:29)
    at jnr.posix.LazyPOSIX.umask(LazyPOSIX.java:279)
    at org.jruby.RubyFile.getUmaskSafe(RubyFile.java:1549)
    at org.jruby.RubyFile.sysopenInternal19(RubyFile.java:1266)
    at org.jruby.RubyFile.fileOpenGeneric(RubyFile.java:1207)
    at org.jruby.RubyIO.ioOpen(RubyIO.java:3833)
    at org.jruby.RubyIO.openKeyArgs(RubyIO.java:3814)
    at org.jruby.RubyIO.read19(RubyIO.java:3776)
    at org.jruby.RubyIO.read19(RubyIO.java:3955)
    at org.jruby.RubyIO$INVOKER$s$0$3$read19.call(RubyIO$INVOKER$s$0$3$read19.gen)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:218)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:214)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:346)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:204)
    at org.jruby.ast.CallTwoArgNode.interpret(CallTwoArgNode.java:59)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.IfNode.interpret(IfNode.java:116)
    at org.jruby.ast.LocalAsgnNode.interpret(LocalAsgnNode.java:123)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
    at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
    at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:182)
    at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:198)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:326)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:170)
    at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
    at org.jruby.ast.DAsgnNode.interpret(DAsgnNode.java:110)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
    at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
    at org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206)
    at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:157)
    at org.jruby.runtime.Interpreted19Block.yieldSpecific(Interpreted19Block.java:135)
    at org.jruby.runtime.Block.yieldSpecific(Block.java:120)
    at org.jruby.ast.YieldOneNode.interpret(YieldOneNode.java:35)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
    at org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206)
    at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:157)
    at org.jruby.runtime.Block.yield(Block.java:142)
    at org.jruby.RubyArray.eachCommon(RubyArray.java:1606)
    at org.jruby.RubyArray.each(RubyArray.java:1613)
    at org.jruby.RubyArray$INVOKER$i$0$0$each.call(RubyArray$INVOKER$i$0$0$each.gen)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:316)
    at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:145)
    at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:154)
    at org.jruby.ast.CallNoArgBlockNode.interpret(CallNoArgBlockNode.java:64)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.evaluator.ASTInterpreter.INTERPRET_BLOCK(ASTInterpreter.java:112)
    at org.jruby.runtime.Interpreted19Block.evalBlockBody(Interpreted19Block.java:206)
    at org.jruby.runtime.Interpreted19Block.yield(Interpreted19Block.java:157)
    at org.jruby.runtime.Block.yield(Block.java:142)
    at org.jruby.RubyArray.eachCommon(RubyArray.java:1606)
    at org.jruby.RubyArray.each(RubyArray.java:1613)
    at org.jruby.RubyArray$INVOKER$i$0$0$each.call(RubyArray$INVOKER$i$0$0$each.gen)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:316)
    at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:145)
    at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:154)
    at org.jruby.ast.CallNoArgBlockNode.interpret(CallNoArgBlockNode.java:64)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
    at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:204)
    at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:206)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:336)
    at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:179)
    at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:188)
    at org.jruby.ast.FCallOneArgBlockNode.interpret(FCallOneArgBlockNode.java:34)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
    at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:204)
    at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:206)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:336)
    at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:179)
    at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:188)
    at org.jruby.ast.FCallOneArgBlockNode.interpret(FCallOneArgBlockNode.java:34)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.evaluator.ASTInterpreter.INTERPRET_METHOD(ASTInterpreter.java:74)
    at org.jruby.internal.runtime.methods.InterpretedMethod.call(InterpretedMethod.java:139)
    at org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:182)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:306)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:136)
    at org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:60)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
    at org.jruby.ast.RootNode.interpret(RootNode.java:129)
    at org.jruby.evaluator.ASTInterpreter.INTERPRET_ROOT(ASTInterpreter.java:121)
    at org.jruby.Ruby.runInterpreter(Ruby.java:838)
    at org.jruby.Ruby.loadFile(Ruby.java:2766)
    at org.jruby.runtime.load.ExternalScript.load(ExternalScript.java:66)
    at org.jruby.runtime.load.LoadService.tryLoadingLibraryOrScript(LoadService.java:959)
    at org.jruby.runtime.load.LoadService.smartLoadInternal(LoadService.java:573)
    at org.jruby.runtime.load.LoadService.requireCommon(LoadService.java:457)
    at org.jruby.runtime.load.LoadService.require(LoadService.java:421)
    at org.jruby.RubyKernel.requireCommon(RubyKernel.java:1091)
    at org.jruby.RubyKernel.require19(RubyKernel.java:1085)
    at org.jruby.RubyKernel$INVOKER$s$1$0$require19.call(RubyKernel$INVOKER$s$1$0$require19.gen)
    at org.jruby.internal.runtime.methods.JavaMethod$JavaMethodOneOrNBlock.call(JavaMethod.java:350)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:326)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:170)
    at org.jruby.ast.FCallOneArgNode.interpret(FCallOneArgNode.java:36)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.RescueNode.executeBody(RescueNode.java:221)
    at org.jruby.ast.RescueNode.interpret(RescueNode.java:116)
    at org.jruby.ast.BeginNode.interpret(BeginNode.java:83)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.RootNode.interpret(RootNode.java:129)
    at org.jruby.evaluator.ASTInterpreter.INTERPRET_ROOT(ASTInterpreter.java:121)
    at org.jruby.Ruby.runInterpreter(Ruby.java:838)
    at org.jruby.Ruby.loadFile(Ruby.java:2766)
    at org.jruby.runtime.load.ExternalScript.load(ExternalScript.java:66)
    at org.jruby.runtime.load.LoadService.load(LoadService.java:359)
    at org.jruby.RubyKernel.loadCommon(RubyKernel.java:1109)
    at org.jruby.RubyKernel.load19(RubyKernel.java:1101)
    at org.jruby.RubyKernel$INVOKER$s$0$1$load19.call(RubyKernel$INVOKER$s$0$1$load19.gen)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:210)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:206)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:326)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:170)
    at org.jruby.ast.FCallOneArgNode.interpret(FCallOneArgNode.java:36)
    at org.jruby.ast.IfNode.interpret(IfNode.java:118)
    at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:105)
    at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
    at org.jruby.ast.RootNode.interpret(RootNode.java:129)
    at org.jruby.evaluator.ASTInterpreter.INTERPRET_ROOT(ASTInterpreter.java:121)
    at org.jruby.Ruby.runInterpreter(Ruby.java:838)
    at org.jruby.Ruby.loadFile(Ruby.java:2766)
    at org.jruby.runtime.load.ExternalScript.load(ExternalScript.java:66)
    at org.jruby.runtime.load.LoadService.loadFromClassLoader(LoadService.java:385)
    at org.jruby.Ruby.initRubyKernel(Ruby.java:1730)
    at org.jruby.Ruby.init(Ruby.java:1209)
    at org.jruby.Ruby.newInstance(Ruby.java:278)
    at org.jruby.Main.internalRun(Main.java:258)
    at org.jruby.Main.run(Main.java:217)
    at org.jruby.Main.main(Main.java:197)
Caused by: java.lang.ExceptionInInitializerError
    at java.lang.J9VMInternals.initialize(J9VMInternals.java:258)
    at jnr.ffi.provider.jffi.NativeRuntime.getInstance(NativeRuntime.java:49)
    at jnr.ffi.provider.jffi.Provider.<init>(Provider.java:29)
    at java.lang.J9VMInternals.newInstanceImpl(Native Method)
    at java.lang.Class.newInstance(Class.java:1616)
    at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.getInstance(FFIProvider.java:60)
    at jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.<clinit>(FFIProvider.java:49)
    at java.lang.J9VMInternals.initializeImpl(Native Method)
    at java.lang.J9VMInternals.initialize(J9VMInternals.java:236)
    at jnr.ffi.provider.FFIProvider.getSystemProvider(FFIProvider.java:35)
    at jnr.ffi.Runtime$SingletonHolder.<clinit>(Runtime.java:85)
    at java.lang.J9VMInternals.initializeImpl(Native Method)
    at java.lang.J9VMInternals.initialize(J9VMInternals.java:236)
    at jnr.ffi.Runtime.getSystemRuntime(Runtime.java:70)
    at jnr.posix.NativePOSIX.<init>(NativePOSIX.java:9)
    at jnr.posix.BaseNativePOSIX.<init>(BaseNativePOSIX.java:35)
    ... 139 more
Caused by: java.lang.IllegalStateException: Cause already initialized
    at java.lang.Throwable.initCause(Throwable.java:344)
    at com.kenai.jffi.Type$Builtin.lookupTypeInfo(Type.java:252)
    at com.kenai.jffi.Type$Builtin.getTypeInfo(Type.java:237)
    at com.kenai.jffi.Type.resolveSize(Type.java:155)
    at com.kenai.jffi.Type.size(Type.java:138)
    at jnr.ffi.provider.jffi.NativeRuntime$TypeDelegate.size(NativeRuntime.java:178)
    at jnr.ffi.provider.AbstractRuntime.<init>(AbstractRuntime.java:48)
    at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:57)
    at jnr.ffi.provider.jffi.NativeRuntime.<init>(NativeRuntime.java:41)
    at jnr.ffi.provider.jffi.NativeRuntime$SingletonHolder.<clinit>(NativeRuntime.java:53)
    at java.lang.J9VMInternals.initializeImpl(Native Method)
    at java.lang.J9VMInternals.initialize(J9VMInternals.java:236)
    ... 154 more
LoadError: Could not load FFI Provider: (NotImplementedError) FFI not available: null
 See http://jira.codehaus.org/browse/JRUBY-4583
  require at org/jruby/RubyKernel.java:1085
  require at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55
   (root) at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/ffi/ffi.rb:69
  require at org/jruby/RubyKernel.java:1085
   (root) at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:1
  require at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55
  require at org/jruby/RubyKernel.java:1085
   (root) at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/ffi.rb:1
   (root) at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:1
  require at file:/opt/logstash/vendor/jar/jruby-complete-1.7.11.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55
  require at org/jruby/RubyKernel.java:1085
     LibC at /opt/logstash/lib/logstash/util/prctl.rb:4
   (root) at /opt/logstash/lib/logstash/util/prctl.rb:3
     main at /opt/logstash/lib/logstash/runner.rb:79
   (root) at /opt/logstash/lib/logstash/runner.rb:215
colinsurprenant commented 9 years ago

@ralphbellofatto what version of logstash are you using?

ralphbellofatto commented 9 years ago

we are running version: logstash-1.4.2-1_2c0f5a1.noarch.rpm

Ralph Bellofatto IBM TJ Watson Research 1-914-945-3321 ralphbel@us.ibm.com

ph commented 9 years ago

This is an issue with JRuby and jffi, from what I've seen in this thread https://github.com/jruby/jruby/issues/1389, this discussion contains some information specific to linux ppc64 (big endian).

The binaries for the both ppc64 and ppc64le where merged in jffi https://github.com/jnr/jffi/pull/12 but JRuby doesn't include this particular version in in their latest release (1.7.19), from the previous discussion they hope to include them in .20.

ph commented 9 years ago

Logstash 1.4.2, ships with JRuby 1.7.11. As soon as the new version is release the master branch will support it.

ralphbellofatto commented 9 years ago

From this issue: jruby/jruby#1389 it sounds like there may be a way for me to temporarly patch things so that we can continue the evaulation and then pick up a new version of logstash once this is fixed.

Are there directions somewhere on how to patch in new binaries into an existing jar file?

Ralph Bellofatto IBM TJ Watson Research 1-914-945-3321 ralphbel@us.ibm.com

ph commented 9 years ago

@ralphbellofatto The changes were committed in a JRuby branch a few hours ago, you could you try to build a new jruby-complete jar with that branch?https://github.com/jruby/jruby/issues/1389#issuecomment-76991439

I've never compiled JRuby myself but their instruction look complete https://github.com/jruby/jruby/blob/master/BUILDING.md

ralphbellofatto commented 9 years ago

hm...

Getting maven for ppc64 appears to be a bit of a problem right now. Do you know where I can get an rpm for RHEL6.5, ppc64 architecture?

Ralph Bellofatto IBM TJ Watson Research 1-914-945-3321 ralphbel@us.ibm.com

From: Pier-Hugues Pellerin notifications@github.com To: elasticsearch/logstash logstash@noreply.github.com Cc: Ralph Bellofatto/Watson/IBM@IBMUS Date: 03/03/2015 03:41 PM Subject: Re: [logstash] logstash running on linux ppc64 crashes on startup. (#2747)

@ralphbellofatto The changes were committed in a JRuby branch a few hours ago, you could you try to build a new jruby-complete jar with that branch? jruby/jruby#1389 (comment)

I've never compiled JRuby myself but their instruction look complete https://github.com/jruby/jruby/blob/master/BUILDING.md

— Reply to this email directly or view it on GitHub.

ralphbellofatto commented 9 years ago

Not too much luck compiling this.

I issued the "vn -Pcomplete " and it issued thousands of error messages. starting with this:

[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ jruby-core
---
[INFO] Compiling 1031 source files
to /home/ralphbel/jruby/jruby-jruby-1_7/core/target/classes
[WARNING] Unable to autodetect 'javac' path, using 'javac' from the
environment.
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] no more tokens - could not parse error message: ----------
1. WARNING
in /home/ralphbel/jruby/jruby-jruby-1_7/core/src/main/java/org/jruby/util/unsafe/UnsafeHolder.java
 (at line 47)
        Class unsafeClass = Class.forName("sun.misc.Unsafe");
        ^^^^^

[ERROR] no more tokens - could not parse error message: Class is a raw
type. References to generic type Class<T> should be parameterized
----------
2. WARNING
in /home/ralphbel/jruby/jruby-jruby-1_7/core/src/main/java/org/jruby/util/unsafe/UnsafeHolder.java
 (at line 83)
        public static long fieldOffset(Class clazz, String name) {
ph commented 9 years ago

I am sorry, I am not really familiar with the RPM ecosystem, specially the ppc one. .20 is just right of the corner, I'll give it a shot asap.

ralphbellofatto commented 9 years ago

made some more progress by updating to a newer jdk and ant version.

However, i'm now getting a BUILD_FAILURE and no explanation as to why... It just seems to say jruby-core failed, but no real explanation why.

ralphbel@bgxcat jruby-jruby-1_7]$ mvn -Pcomplete 2>&1 |
tee ../jruby.build.log
[INFO] Scanning for projects...
[INFO]
------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] JRuby
[INFO] JRuby Core
[INFO] JRuby Ext
[INFO] JRuby Readline
[INFO] JRuby Ripper
[INFO] JRuby Lib Setup
[INFO] JRuby Artifacts
[INFO] JRuby Stdlib
[INFO] JRuby Complete
[INFO]
[INFO]
------------------------------------------------------------------------
[INFO] Building JRuby 1.7.20-SNAPSHOT
[INFO]
------------------------------------------------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:1.0:enforce (enforce-maven) @ jruby-parent
---
[INFO]
[INFO] --- maven-install-plugin:2.4:install (default-install) @
jruby-parent ---
[INFO] Installing /home/ralphbel/jruby/jruby-jruby-1_7/pom.xml
to /gsa/yktgsa/home/r/a/ralphbel/.m2/repository/org/jruby/jruby-parent/1.7.20-SNAPSHOT/jruby-parent-1.7.20-SNAPSHOT.pom
[INFO]
[INFO]
------------------------------------------------------------------------
[INFO] Building JRuby Core 1.7.20-SNAPSHOT
[INFO]
------------------------------------------------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:1.0:enforce (enforce-maven) @ jruby-core
---
[INFO]
[INFO] --- properties-maven-plugin:1.0-alpha-2:read-project-properties
(properties) @ jruby-core ---
[WARNING] Ignoring missing properties
file: /home/ralphbel/jruby/jruby-jruby-1_7/core/../build.properties
[INFO]
[INFO] --- buildnumber-maven-plugin:1.2:create (jruby-revision) @
jruby-core ---
[INFO] Checking for local modifications: skipped.
[INFO] Updating project files from SCM: skipped.
[INFO] ShortRevision tag detected. The value is '7'.
[INFO] Executing: /bin/sh -c cd /home/ralphbel/jruby/jruby-jruby-1_7/core
&& git rev-parse --verify --short=7 HEAD
[INFO] Working directory: /home/ralphbel/jruby/jruby-jruby-1_7/core
[INFO] Storing buildNumber: null at timestamp: 1425425250957
[INFO] ShortRevision tag detected. The value is '7'.
[INFO] Executing: /bin/sh -c cd /home/ralphbel/jruby/jruby-jruby-1_7/core
&& git rev-parse --verify --short=7 HEAD
[INFO] Working directory: /home/ralphbel/jruby/jruby-jruby-1_7/core
[INFO] Storing buildScmBranch: UNKNOWN_BRANCH
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
jruby-core ---
[INFO] Using 'utf-8' encoding to copy filtered resources.
[INFO] Copying 44 resources
[INFO] Copying 1 resource
[INFO] Copying 1 resource
to /home/ralphbel/jruby/jruby-jruby-1_7/core/src/main/java
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (anno) @ jruby-core ---
[INFO] Compiling 7 source files
to /home/ralphbel/jruby/jruby-jruby-1_7/core/target/classes
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ jruby-core
---
[INFO] Compiling 1356 source files
to /home/ralphbel/jruby/jruby-jruby-1_7/core/target/classes
[INFO]
------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] JRuby .............................................. SUCCESS
[  1.168 s]
[INFO] JRuby Core ......................................... FAILURE
[  6.385 s]
[INFO] JRuby Ext .......................................... SKIPPED
[INFO] JRuby Readline ..................................... SKIPPED
[INFO] JRuby Ripper ....................................... SKIPPED
[INFO] JRuby Lib Setup .................................... SKIPPED
[INFO] JRuby Artifacts .................................... SKIPPED
[INFO] JRuby Stdlib ....................................... SKIPPED
[INFO] JRuby Complete ..................................... SKIPPED
[INFO]
------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 10.252 s
[INFO] Finished at: 2015-03-03T18:27:34-05:00
[INFO] Final Memory: 15M/24M
[INFO]
------------------------------------------------------------------------
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-compiler-plugin:3.1:compile
(default-compile) on project jruby-core: Compilation failure -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions,
please read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the
command
[ERROR]   mvn <goals> -rf :jruby-core
ralphbellofatto commented 9 years ago

to try to eliminate the IBM JRE and JDK from this problem i brought provisioned RHEL7 system and installed openjdk on it.

my java version shows:

[root@dccxcat jruby-jruby-1_7]# java -version
java version "1.7.0_75"
OpenJDK Runtime Environment (rhel-2.5.4.2.el7_0-ppc64 u75-b13)
OpenJDK 64-Bit Server VM (build 24.75-b04, mixed mode)

my java home directory is:

JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.75-2.5.4.2.el7_0.ppc64

I run:

 mvn -Pcomplete

and I get still get the following error messages:

INFO] Reactor Summary:
[INFO]
[INFO] JRuby .............................................. SUCCESS [  0.342 s]
[INFO] JRuby Core ......................................... FAILURE [ 10.921 s]
[INFO] JRuby Ext .......................................... SKIPPED
[INFO] JRuby Readline ..................................... SKIPPED
[INFO] JRuby Ripper ....................................... SKIPPED
[INFO] JRuby Lib Setup .................................... SKIPPED
[INFO] JRuby Artifacts .................................... SKIPPED
[INFO] JRuby Stdlib ....................................... SKIPPED
[INFO] JRuby Complete ..................................... SKIPPED
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12.334 s
[INFO] Finished at: 2015-03-13T10:48:43-04:00
[INFO] Final Memory: 24M/418M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) on project jruby-core: Compilation failure: Compilation failure:
[ERROR] /home/ralphbel/jruby/jruby-jruby-1_7/core/src/main/java/org/jruby/util/unsafe/UnsafeHolder.java:[96,10] cannot find symbol
[ERROR] symbol:   method fullFence()
[ERROR] location: variable U of type sun.misc.Unsafe
[ERROR] /home/ralphbel/jruby/jruby-jruby-1_7/core/src/main/java/org/jruby/util/unsafe/UnsafeHolder.java:[100,10] cannot find symbol
[ERROR] symbol:   method loadFence()
[ERROR] location: variable U of type sun.misc.Unsafe
[ERROR] /home/ralphbel/jruby/jruby-jruby-1_7/core/src/main/java/org/jruby/util/unsafe/UnsafeHolder.java:[104,10] cannot find symbol
[ERROR] symbol:   method storeFence()
[ERROR] location: variable U of type sun.misc.Unsafe
[ERROR] -> [Help 1]

So... how do we resolve this on an openjdk installation so i can compile.

Also any ETA on the availability of jruby .20,

wiibaa commented 9 years ago

@ralphbellofatto while waiting for jruby .20 maybe you could give a try to logstash-1.5.0rc2 (that use jruby-bin instead of complete jar) and replace vendor/jruby with jruby-1.7.20 snapshot build from http://ci.jruby.org/, it seems to contains the necessary jffi library for your OS

ralphbellofatto commented 9 years ago

Ok,

We have a little progress, by adding the hand compiled jruby from source:

[root@dccxcat logstash-1.5.0.rc2]# bin/logstash -e 'input { stdin { } } output { stdout { codec => rubydebug } }'
Mar 16, 2015 10:54:56 AM jnr.ffi.provider.jffi.NativeRuntime buildNativeTypeAliases
SEVERE: failed to load type aliases: java.lang.ClassNotFoundException: jnr/ffi/provider/jffi/platform/ppc64/linux/TypeAliases
Hello There
Logstash startup completed
{
       "message" => "Hello There",
      "@version" => "1",
    "@timestamp" => "2015-03-16T14:55:05.169Z",
          "host" => "dccxcat"
}
oh boogers
{
       "message" => "oh boogers",
      "@version" => "1",
    "@timestamp" => "2015-03-16T14:55:15.561Z",
          "host" => "dccxcat"
}

It seems to be basically functional, I guess until it tries to do something where it is going to need that FFI provider class.

ralphbellofatto commented 9 years ago

While the above works for the stdin-> stdout script, logstash fails on ppc64 when I try to have digest a file with timestamps in it.

We installed RC2 as follows:

mkdir /opt/logstash
http://download.elasticsearch.org/logstash/logstash/logstash-1.5.0.rc2.tar.gz
tar xzf logstash-1.5.0.rc2.tar.gz
cd logstash-1.5.0.rc2
cp /home/ralphbel/jruby/jruby-jruby-1_7/lib/jruby.jar ./vendor/jruby/lib/jruby.jar

and then we constructed the following script

#!/bin/bash
# simple log stash test wrapper...

logstashdir=/opt/logstash-1.5.0.rc2/bin/
logstash_scr=/tmp/logstash_test.scr
logstash_data=/tmp/logstash_test.data

cat >$logstash_scr <<EOF
input {
  file {
    path => "$logstash_data"
    start_position => beginning
    sincedb_path => "/dev/null" 
  }
}
output {
  stdout { codec => rubydebug }
}

EOF

cat >$logstash_data <<"EOF"
Mar 15 03:35:02 dccxcat run-parts(/etc/cron.daily)[89799]: finished logrotate
EOF

cat $logstash_scr
/opt/logstash-1.5.0.rc2/bin/logstash -f $logstash_scr

rm $logstash_scr
rm $logstash_data

This results in the following output and crash:

[dccxcat logstash]# ./ltest.sh
input {
  file {
    path => "/tmp/logstash_test.data"
    start_position => beginning
    sincedb_path => "/dev/null"
  }
}
output {
  stdout { codec => rubydebug }
}

Mar 16, 2015 1:00:39 PM jnr.ffi.provider.jffi.NativeRuntime buildNativeTypeAliases
SEVERE: failed to load type aliases: java.lang.ClassNotFoundException: jnr/ffi/provider/jffi/platform/ppc64/linux/TypeAliases
Logstash startup completed
Logstash shutdown completed
NotImplementedError: stat.st_dev unsupported or native support failed to load
       dev_major at org/jruby/RubyFileStat.java:201
  _discover_file at /opt/logstash-1.5.0.rc2/vendor/bundle/jruby/1.9/gems/filewatch-0.6.1/lib/filewatch/watch.rb:150
            each at org/jruby/RubyArray.java:1613
  _discover_file at /opt/logstash-1.5.0.rc2/vendor/bundle/jruby/1.9/gems/filewatch-0.6.1/lib/filewatch/watch.rb:132
           watch at /opt/logstash-1.5.0.rc2/vendor/bundle/jruby/1.9/gems/filewatch-0.6.1/lib/filewatch/watch.rb:38
            tail at /opt/logstash-1.5.0.rc2/vendor/bundle/jruby/1.9/gems/filewatch-0.6.1/lib/filewatch/tail.rb:68
             run at /opt/logstash-1.5.0.rc2/vendor/bundle/jruby/1.9/gems/logstash-input-file-0.1.6/lib/logstash/inputs/file.rb:133
            each at org/jruby/RubyArray.java:1613
             run at /opt/logstash-1.5.0.rc2/vendor/bundle/jruby/1.9/gems/logstash-input-file-0.1.6/lib/logstash/inputs/file.rb:133
     inputworker at /opt/logstash-1.5.0.rc2/lib/logstash/pipeline.rb:174
     start_input at /opt/logstash-1.5.0.rc2/lib/logstash/pipeline.rb:168
            call at org/jruby/RubyProc.java:290
            call at org/jruby/RubyProc.java:228
ralphbellofatto commented 9 years ago

I think I finally figured out what this was trying to tell me:

We did:

wget http://ci.jruby.org/snapshots/jruby-1_7/jruby-dist-1.7.20-SNAPSHOT-bin.tar.gz
cd /home/ralphbel/jruby
tar xvxf http://ci.jruby.org/snapshots/jruby-1_7/jruby-dist-1.7.20-SNAPSHOT-bin.tar.gz
     cd /opt/logstash-1.5.0.rc2.patched/vendor/jruby
     rm -rf *
     cp -R /home/ralphbel/jruby/jruby-1.7.20-SNAPSHOT/* .

And then.

we did:

create a script ltest-stdout.sh with:

#!/bin/bash
# simple log stash test wrapper...

logstashdir=/opt/logstash-1.5.0.rc2/bin/
set -x
$logstashdir/logstash -e 'input { stdin { } } output { stdout { codec => rubydebug } }'

when we execute it we get:

[root@dccxcat logstash-test]# ./ltest-stdout.sh
+ /opt/logstash-1.5.0.rc2/bin//logstash -e 'input { stdin { } } output { stdout { codec => rubydebug } }'
LoadError: Could not load FFI Provider: (NotImplementedError) FFI not available: null
 See http://jira.codehaus.org/browse/JRUBY-4583
  require at org/jruby/RubyKernel.java:1071
   (root) at /opt/logstash-1.5.0.rc2/vendor/jruby/lib/ruby/shared/ffi/ffi.rb:69
  require at org/jruby/RubyKernel.java:1071
   (root) at /opt/logstash-1.5.0.rc2/vendor/jruby/lib/ruby/shared/ffi.rb:1
  require at org/jruby/RubyKernel.java:1071
   (root) at /opt/logstash-1.5.0.rc2/vendor/jruby/lib/ruby/shared/ffi.rb:1
   (root) at /opt/logstash-1.5.0.rc2/lib/logstash/util/prctl.rb:1
  require at org/jruby/RubyKernel.java:1071
     LibC at /opt/logstash-1.5.0.rc2/lib/logstash/util/prctl.rb:4
     main at /opt/logstash-1.5.0.rc2/lib/logstash/runner.rb:39
   (root) at /opt/logstash-1.5.0.rc2/lib/logstash/runner.rb:167

The simple filestat test above:

also produceds:

[root@dccxcat logstash-test]# ../jruby/jruby-1.7.20-SNAPSHOT/bin/jruby testfs.rb
File.stat called
Mar 19, 2015 9:59:53 AM jnr.ffi.provider.jffi.NativeRuntime buildNativeTypeAliases
SEVERE: failed to load type aliases: java.lang.ClassNotFoundException: jnr/ffi/provider/jffi/platform/ppc64/linux/TypeAliases
File.stat returned

So it looks like jruby has handled the problem of the ../jruby/jruby-1.7.20-SNAPSHOT/lib/jni/ppc64-Linux/libjffi-1.2.so problem.

but, there is now a problem in the jnr/ffi module.

ralphbellofatto commented 9 years ago

create a script ltest-stdout.sh with:

#!/bin/bash
# simple log stash test wrapper...

logstashdir=/opt/logstash-1.5.0.rc2.patched/bin/
set -x
$logstashdir/logstash -e 'input { stdin { } } output { stdout { codec => rubydebug } }'

when we execute it we get:

@dccxcat logstash-test]# ./ltest-stdout.sh
rm: missing operand
Try 'rm --help' for more information.
+ /opt/logstash-1.5.0.rc2.patched/bin//logstash -e 'input { stdin { } } output { stdout { codec => rubydebug } }'
Mar 19, 2015 10:17:11 AM jnr.ffi.provider.jffi.NativeRuntime buildNativeTypeAliases
SEVERE: failed to load type aliases: java.lang.ClassNotFoundException: jnr/ffi/provider/jffi/platform/ppc64/linux/TypeAliases
Logstash startup completed
hello
{
       "message" => "hello",
      "@version" => "1",
    "@timestamp" => "2015-03-19T14:17:26.787Z",
          "host" => "dccxcat"
}
^CSIGINT received. Shutting down the pipeline. {:level=>:warn}

The simple filestat test above:

also produced:

[root@dccxcat logstash-test]# ../jruby/jruby-1.7.20-SNAPSHOT/bin/jruby testfs.rb
File.stat called
Mar 19, 2015 9:59:53 AM jnr.ffi.provider.jffi.NativeRuntime buildNativeTypeAliases
SEVERE: failed to load type aliases: java.lang.ClassNotFoundException: jnr/ffi/provider/jffi/platform/ppc64/linux/TypeAliases
File.stat returned

So it looks like jruby has handled the problem of the ../jruby/jruby-1.7.20-SNAPSHOT/lib/jni/ppc64-Linux/libjffi-1.2.so problem.

but, there is now a problem in the jnr/ffi module.

ralphbellofatto commented 9 years ago

this script

logstash-test]# cat ltest-file.sh
#!/bin/bash
# simple log stash test wrapper...

logstashdir=/opt/logstash-1.5.0.rc2.patched/bin/
logstash_scr=/tmp/logstash_test.scr
logstash_data=/tmp/logstash_test.data

cat >$logstash_scr <<EOF
input {
  file {
    path => "$logstash_data"
    start_position => beginning
    sincedb_path => "/dev/null"
  }
}
output {
  stdout { codec => rubydebug }
}

EOF

cat >$logstash_data <<"EOF"
Mar 15 03:35:02 dccxcat run-parts(/etc/cron.daily)[89799]: finished logrotate
EOF

cat $logstash_scr
set -x
$logstashdir/logstash -f $logstash_scr
rm $logstash_scr
rm $logstash_data

produces

ogstash-test]# ./ltest-file.sh
input {
  file {
    path => "/tmp/logstash_test.data"
    start_position => beginning
    sincedb_path => "/dev/null"
  }
}
output {
  stdout { codec => rubydebug }
}

+ /opt/logstash-1.5.0.rc2.patched/bin//logstash -f /tmp/logstash_test.scr
Mar 19, 2015 10:19:51 AM jnr.ffi.provider.jffi.NativeRuntime buildNativeTypeAliases
SEVERE: failed to load type aliases: java.lang.ClassNotFoundException: jnr/ffi/provider/jffi/platform/ppc64/linux/TypeAliases
Logstash startup completed
Logstash shutdown completed
NotImplementedError: stat.st_dev unsupported or native support failed to load
       dev_major at org/jruby/RubyFileStat.java:201
  _discover_file at /opt/logstash-1.5.0.rc2.patched/vendor/bundle/jruby/1.9/gems/filewatch-0.6.1/lib/filewatch/watch.rb:150
            each at org/jruby/RubyArray.java:1613
  _discover_file at /opt/logstash-1.5.0.rc2.patched/vendor/bundle/jruby/1.9/gems/filewatch-0.6.1/lib/filewatch/watch.rb:132
           watch at /opt/logstash-1.5.0.rc2.patched/vendor/bundle/jruby/1.9/gems/filewatch-0.6.1/lib/filewatch/watch.rb:38
            tail at /opt/logstash-1.5.0.rc2.patched/vendor/bundle/jruby/1.9/gems/filewatch-0.6.1/lib/filewatch/tail.rb:68
             run at /opt/logstash-1.5.0.rc2.patched/vendor/bundle/jruby/1.9/gems/logstash-input-file-0.1.6/lib/logstash/inputs/file.rb:133
            each at org/jruby/RubyArray.java:1613
             run at /opt/logstash-1.5.0.rc2.patched/vendor/bundle/jruby/1.9/gems/logstash-input-file-0.1.6/lib/logstash/inputs/file.rb:133
     inputworker at /opt/logstash-1.5.0.rc2.patched/lib/logstash/pipeline.rb:174
     start_input at /opt/logstash-1.5.0.rc2.patched/lib/logstash/pipeline.rb:168
+ rm /tmp/logstash_test.scr
+ rm /tmp/logstash_test.data
fenglixa commented 9 years ago

Installed Logstash 1.5.0.1 on ppc64le (IBM Power Linux), but logstash still cannot startup:

rpm -qa |grep logstash

logstash-1.5.0-1.noarch

/opt/logstash/bin/logstash -f /etc/logstash/conf.d -l /var/log/logstash/logstash.log

io/console not supported; tty will not be manipulated LoadError: Unsupported platform: unknown-linux require at org/jruby/RubyKernel.java:1071 (root) at /opt/logstash/vendor/jruby/lib/ruby/shared/ffi/ffi.rb:69 require at org/jruby/RubyKernel.java:1071 (root) at /opt/logstash/vendor/jruby/lib/ruby/shared/ffi.rb:1 require at org/jruby/RubyKernel.java:1071 (root) at /opt/logstash/vendor/jruby/lib/ruby/shared/ffi.rb:1 (root) at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.0-java/lib/logstash/util/prctl.rb:1 require at org/jruby/RubyKernel.java:1071 LibC at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.0-java/lib/logstash/util/prctl.rb:4 (root) at /opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.0-java/lib/logstash/util/prctl.rb:3 (root) at /opt/logstash/lib/bootstrap/environment.rb:48

colinsurprenant commented 9 years ago

As discussed in jnr/jffi#5 with @fenglixa this should be solved with JRuby 1.7.20. I will describe how to generate a logstash package with updated JRuby version from 1.5.0. Upcoming 1.5.1 or 1.6.0 (whichever comes first) will include JRuby 1.7.20.

colinsurprenant commented 9 years ago

clone logstash repo

git clone https://github.com/elastic/logstash.git

switch to v1.5.0 tag

cd logstash
git checkout v1.5.0

edit rakelib/vendor.rake and change the jruby reference a the top to

"jruby" => { "version" => "1.7.20", "sha1" => "3c11f01d38b9297cef2c281342f8bb799772e481" },

create tar.gz package

rake bootstrap
rake artifact:tar

this will create build/logstash-1.5.0.tar.gz.

You can also create .zip, .deb or .rpm packages using

rake artifact:zip
rake artifact:rpm
rake artifact:deb
fenglixa commented 9 years ago

Logstash could start-up successfully on pcc64le after rebuilding logstash with jruby1.7.20. Thanks @colinsurprenant

ps -ewf |grep logstash

root 1712 5189 0 05:03 pts/3 00:00:00 grep --color=auto logstash root 24337 12081 8 03:08 pts/1 00:09:35 /opt/pcm/web-portal/perf/jre/linux-ppc64le/bin/java -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Xmx500m -Xss2048k -Djffi.boot.library.path=/opt/logstash/vendor/jruby/lib/jni -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -Djava.awt.headless=true -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Xbootclasspath/a:/opt/logstash/vendor/jruby/lib/jruby.jar -classpath : -Djruby.home=/opt/logstash/vendor/jruby -Djruby.lib=/opt/logstash/vendor/jruby/lib -Djruby.script=jr

colinsurprenant commented 9 years ago

ah! great! :thumbsup:

colinsurprenant commented 9 years ago

So I will close this issue which will be fixed by #3312 plus the workaround above until we release 1.5.1/1.6.0.

maheshwarishikha commented 9 years ago

I am also trying to build logstash with jruby 1.7.20 on RHEL7 ppc64le architecture . Did following steps:

Looks like jruby issue but jruby 1.7.20 should work fine on ppc64le. What could be the reason... Kindly help.