search

elastic / logstash

Logstash - transport and process your logs, events, or other data
https://www.elastic.co/products/logstash
Other
14.28k stars 3.51k forks source link

Error with running artifact:tar task #2744

Closed electrical closed 9 years ago

electrical commented 9 years ago

[plugin:install-default] Installing default plugins Installing logstash-output-zeromq, logstash-codec-collectd, logstash-output-xmpp, logstash-codec-dots, logstash-codec-edn, logstash-codec-edn_lines, logstash-codec-fluent, logstash-codec-es_bulk, logstash-codec-graphite, logstash-codec-json, logstash-codec-json_lines, logstash-codec-line, logstash-codec-msgpack, logstash-codec-multiline, logstash-codec-netflow, logstash-codec-oldlogstashjson, logstash-codec-plain, logstash-codec-rubydebug, logstash-filter-anonymize, logstash-filter-checksum, logstash-filter-clone, logstash-filter-csv, logstash-filter-date, logstash-filter-dns, logstash-filter-drop, logstash-filter-fingerprint, logstash-filter-geoip, logstash-filter-grok, logstash-filter-json, logstash-filter-kv, logstash-filter-metrics, logstash-filter-multiline, logstash-filter-mutate, logstash-filter-ruby, logstash-filter-sleep, logstash-filter-split, logstash-filter-syslog_pri, logstash-filter-throttle, logstash-filter-urldecode, logstash-filter-useragent, logstash-filter-uuid, logstash-filter-xml, logstash-input-couchdb_changes, logstash-input-elasticsearch, logstash-input-eventlog, logstash-input-exec, logstash-input-file, logstash-input-ganglia, logstash-input-gelf, logstash-input-generator, logstash-input-graphite, logstash-input-imap, logstash-input-irc, logstash-input-kafka, logstash-input-log4j, logstash-input-lumberjack, logstash-input-pipe, logstash-input-rabbitmq, logstash-input-redis, logstash-input-s3, logstash-input-snmptrap, logstash-input-sqs, logstash-input-stdin, logstash-input-syslog, logstash-input-tcp, logstash-input-twitter, logstash-input-udp, logstash-input-unix, logstash-input-xmpp, logstash-input-zeromq, logstash-output-cloudwatch, logstash-output-csv, logstash-output-elasticsearch, logstash-output-email, logstash-output-exec, logstash-output-file, logstash-output-ganglia, logstash-output-gelf, logstash-output-graphite, logstash-output-hipchat, logstash-output-http, logstash-output-irc, logstash-output-juggernaut, logstash-output-kafka, logstash-output-lumberjack, logstash-output-nagios, logstash-output-nagios_nsca, logstash-output-null, logstash-output-opentsdb, logstash-output-pagerduty, logstash-output-pipe, logstash-output-rabbitmq, logstash-output-redis, logstash-output-s3, logstash-output-sns, logstash-output-sqs, logstash-output-statsd, logstash-output-stdout, logstash-output-tcp, logstash-output-udp Installation successful Errno::EBADF: Bad file descriptor - Bad file descriptor rewind at org/jruby/RubyIO.java:1886 save at /home/jenkins/workspace/logstash_regression_15/jdk/JDK7/label/metal-pool/lib/logstash/gemfile.rb:32 close at /home/jenkins/workspace/logstash_regression_15/jdk/JDK7/label/metal-pool/lib/logstash/gemfile.rb:62 execute at /home/jenkins/workspace/logstash_regression_15/jdk/JDK7/label/metal-pool/lib/logstash/pluginmanager/install.rb:90 run at /home/jenkins/workspace/logstash_regression_15/jdk/JDK7/label/metal-pool/vendor/bundle/jruby/1.9/gems/clamp-0.6.4/lib/clamp/command.rb:67 execute at /home/jenkins/workspace/logstash_regression_15/jdk/JDK7/label/metal-pool/vendor/bundle/jruby/1.9/gems/clamp-0.6.4/lib/clamp/subcommand/execution.rb:11 run at /home/jenkins/workspace/logstash_regression_15/jdk/JDK7/label/metal-pool/vendor/bundle/jruby/1.9/gems/clamp-0.6.4/lib/clamp/command.rb:67 run at /home/jenkins/workspace/logstash_regression_15/jdk/JDK7/label/metal-pool/vendor/bundle/jruby/1.9/gems/clamp-0.6.4/lib/clamp/command.rb:132 (root) at /home/jenkins/workspace/logstash_regression_15/jdk/JDK7/label/metal-pool/lib/logstash/pluginmanager.rb:12 rake aborted!

/home/jenkins/workspace/logstash_regression_15/jdk/JDK7/label/metal-pool/rakelib/plugin.rake:7:in install_plugins' /home/jenkins/workspace/logstash_regression_15/jdk/JDK7/label/metal-pool/rakelib/plugin.rake:27:in(root)' org/jruby/RubyProc.java:271:in call' org/jruby/RubyArray.java:1613:ineach' org/jruby/RubyArray.java:1613:in each' org/jruby/RubyArray.java:1613:ineach' org/jruby/RubyArray.java:1613:in each' org/jruby/RubyKernel.java:1087:inload' Tasks: TOP => artifact:tar => artifact:prepare => plugin:install-default (See full trace by running task with --trace) Invoking bundler install... Your Gemfile lists the gem logstash-devutils (>= 0) more than once. You should probably keep only one of them. While it's not a problem now, it could cause errors if you change the version of just one of them later.

jsvd commented 9 years ago

I added

puts @io
puts @io.stat
puts @io.closed?

before the rewind and got

#<File:0x44e1153d>
#<File::Stat dev=0x1000004, ino=21415278, mode=0100644, nlink=1, uid=501, gid=20, rdev=0x0, size=3048, blksize=4096, blocks=8, atime=2015-03-03 12:09:20 +0000, mtime=2015-03-03 12:09:17 +0000, ctime=2015-03-03 12:09:17 +0000>
false
Errno::EBADF: Bad file descriptor - Bad file descriptor
   rewind at org/jruby/RubyIO.java:1886
     save at /Users/joaoduarte/projects/logstash/lib/logstash/gemfile.rb:32
    close at /Users/joaoduarte/projects/logstash/lib/logstash/gemfile.rb:65
  execute at /Users/joaoduarte/projects/logstash/lib/logstash/pluginmanager/install.rb:90
      run at /Users/joaoduarte/projects/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.4/lib/clamp/command.rb:67
  execute at /Users/joaoduarte/projects/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.4/lib/clamp/subcommand/execution.rb:11
      run at /Users/joaoduarte/projects/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.4/lib/clamp/command.rb:67
      run at /Users/joaoduarte/projects/logstash/vendor/bundle/jruby/1.9/gems/clamp-0.6.4/lib/clamp/command.rb:132
   (root) at /Users/joaoduarte/projects/logstash/lib/logstash/pluginmanager.rb:12
rake aborted!

So the file is there and looks OK..debugging further..

jsvd commented 9 years ago

this is the issue, with jruby: https://github.com/jruby/jruby/issues/779#issuecomment-45154930

it's fixed in 9.0.0.0pre1 but not even on 1.7.19.

purbon commented 9 years ago

Can we workaround this somehow? and move forward.

jsvd commented 9 years ago

trying to replace r+ with a+ and rewind

diff --git a/lib/logstash/gemfile.rb b/lib/logstash/gemfile.rb
index 675572e..837f706 100644
--- a/lib/logstash/gemfile.rb
+++ b/lib/logstash/gemfile.rb
@@ -16,7 +16,9 @@ module LogStash
     end

     def self.open(file_path)
-      gemfile = new(::File.new(file_path, "r+"))
+      fd = ::File.new(file_path, "a+")
+      fd.rewind # work around https://github.com/jruby/jruby/issues/779#issuecomment-45154930
+      gemfile = new(fd)
       gemfile.load
       gemfile
     end
jsvd commented 9 years ago

Using "a+" also triggers the errors. next test: removing the extra call to gemfile.save during close

jsvd commented 9 years ago

failed to come up with a quick workaround for the jruby bug, reverted patch and reopened #2626

colinsurprenant commented 9 years ago

since we are reverting we can remove the blocker label and either close this or move it for 1.5.0.

suyograo commented 9 years ago

Since #2626 is opened, and acceptance tests are passing, will close this.