search

elastic / logstash

Logstash - transport and process your logs, events, or other data
https://www.elastic.co/products/logstash
Other
14.17k stars 3.49k forks source link

1.4.2 logstash deps nokogiri build error #1826

Closed cdenneen closed 9 years ago

cdenneen commented 9 years ago

I was able to do 1.5.0-dev bin/logstash deps without issue but when I did a fresh clone of 1.4.2 I received the following errors:

chris@chris-macbook-pro ~/src/github $ git clone http://github.com/elasticsearch/logstash
Cloning into 'logstash'...
remote: Counting objects: 34431, done.
remote: Compressing objects: 100% (190/190), done.
remote: Total 34431 (delta 85), reused 73 (delta 8)
Receiving objects: 100% (34431/34431), 68.33 MiB | 132.00 KiB/s, done.
Resolving deltas: 100% (23388/23388), done.
Checking connectivity... done.
chris@chris-macbook-pro ~/src/github $ cd logstash
chris@chris-macbook-pro ~/src/github/logstash (master)$ git checkout v1.4.2 -b 1.4.2
Switched to a new branch '1.4.2'
chris@chris-macbook-pro ~/src/github/logstash (1.4.2)$ make vendor-jruby
=> Downloading jruby 1.7.11
chris@chris-macbook-pro ~/src/github/logstash (1.4.2)$ ./bin/logstash deps
Fetching and installing gem: bundler (>= 1.3.5)
Successfully installed bundler-1.7.3
1 gem installed
Fetching gem metadata from https://rubygems.org/.......
Fetching additional metadata from https://rubygems.org/..
Resolving dependencies...
Installing minitest 5.4.2
Installing amq-protocol 1.9.2
Installing avl_tree 1.1.3
Installing addressable 2.3.6
Installing thread_safe 0.3.4
Installing i18n 0.6.11
Installing atomic 1.1.16
Installing awesome_print 1.2.0
Installing beefcake 0.3.7
Installing buftok 0.1
Installing bindata 2.1.0
Installing mini_portile 0.6.0
Installing backports 3.6.1
Installing json 1.8.1
Installing builder 3.2.2
Installing cabin 0.6.1
Installing clamp 0.6.3
Installing cinch 2.1.0
Installing coderay 1.1.0
Installing multi_json 1.10.1
Installing edn 1.0.6
Installing diff-lcs 1.2.5
Installing multipart-post 2.0.0
Installing extlib 0.9.16
Installing excon 0.39.6
Installing filewatch 0.5.1
Installing gelfd 0.2.0
Installing gmetric 0.1.3
Installing geoip 1.4.0
Installing http_parser.rb 0.5.3
Installing jls-lumberjack 0.0.20
Installing hitimes 1.2.2
Installing mime-types 1.25.1
Installing polyglot 0.3.5
Installing method_source 0.8.2
Installing murmurhash3 0.1.4
Installing slop 3.6.0
Installing metaclass 0.0.4
Installing rspec-support 3.1.1
Installing redis 3.1.0
Installing rack 1.5.2
Installing simple_oauth 0.2.0
Installing insist 1.0.0
Installing tilt 1.4.1
Installing msgpack 0.5.8
Installing shoulda-context 1.2.1
Using bundler 1.7.3
Installing statsd-ruby 1.2.0
Installing snmp 1.2.0
Installing xml-simple 1.1.4
Installing user_agent_parser 2.1.5
Installing xmpp4r 0.5
Installing gelf 1.3.2
Installing tzinfo 1.2.2
Installing bunny 1.1.9
Installing ffi 1.9.5
Installing ci_reporter 2.0.0
Installing jls-grok 0.10.12
Installing elasticsearch-api 1.0.5
Installing metriks 0.9.9.6
Installing faraday 0.9.0
Installing ftw 0.0.40
Installing http 0.5.1
Installing pry 0.10.1
Installing mocha 1.1.0
Installing rspec-core 3.1.5
Installing rspec-expectations 3.1.2
Installing rspec-mocks 3.1.2
Installing treetop 1.4.15
Installing rack-protection 1.5.3
Installing rufus-scheduler 2.0.24
Installing ffi-rzmq 1.0.0
Installing activesupport 4.1.6
Installing spoon 0.0.4
Installing stud 0.0.17
Installing rbnacl 3.1.2
Installing elasticsearch-transport 1.0.5
Installing rspec 3.1.0
Installing mail 2.5.3
Installing shoulda-matchers 2.7.0
Installing elasticsearch 1.0.5
Installing rumbster 1.1.1
Installing shoulda 3.5.0
Installing sinatra 1.4.5
Installing twitter 5.0.0.rc.1
Building nokogiri using packaged libraries.
Building libxml2-2.8.0 for nokogiri with the following patches applied:
    - 0001-Fix-parser-local-buffers-size-problems.patch
    - 0002-Fix-entities-local-buffers-size-problems.patch
    - 0003-Fix-an-error-in-previous-commit.patch
    - 0004-Fix-potential-out-of-bound-access.patch
    - 0005-Detect-excessive-entities-expansion-upon-replacement.patch
    - 0006-Do-not-fetch-external-parsed-entities.patch
    - 0007-Enforce-XML_PARSER_EOF-state-handling-through-the-pa.patch
    - 0008-Improve-handling-of-xmlStopParser.patch
    - 0009-Fix-a-couple-of-return-without-value.patch
    - 0010-Keep-non-significant-blanks-node-in-HTML-parser.patch
    - 0011-Do-not-fetch-external-parameter-entities.patch
************************************************************************
IMPORTANT!  Nokogiri builds and uses a packaged version of libxml2.

If this is a concern for you and you want to use the system library
instead, abort this installation process and reinstall nokogiri as
follows:

    gem install nokogiri -- --use-system-libraries

If you are using Bundler, tell it to use the option:

    bundle config build.nokogiri --use-system-libraries
    bundle install

However, note that nokogiri does not necessarily support all versions
of libxml2.

For example, libxml2-2.9.0 and higher are currently known to be broken
and thus unsupported by nokogiri, due to compatibility problems and
XPath optimization bugs.
************************************************************************

Gem::Installer::ExtensionBuildError: ERROR: Failed to build gem native extension.

    /System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby extconf.rb
Building nokogiri using packaged libraries.
checking for iconv.h... yes
checking for iconv_open() in iconv.h... no
checking for iconv_open() in -liconv... yes
Building libxml2-2.8.0 for nokogiri with the following patches applied:
    - 0001-Fix-parser-local-buffers-size-problems.patch
    - 0002-Fix-entities-local-buffers-size-problems.patch
    - 0003-Fix-an-error-in-previous-commit.patch
    - 0004-Fix-potential-out-of-bound-access.patch
    - 0005-Detect-excessive-entities-expansion-upon-replacement.patch
    - 0006-Do-not-fetch-external-parsed-entities.patch
    - 0007-Enforce-XML_PARSER_EOF-state-handling-through-the-pa.patch
    - 0008-Improve-handling-of-xmlStopParser.patch
    - 0009-Fix-a-couple-of-return-without-value.patch
    - 0010-Keep-non-significant-blanks-node-in-HTML-parser.patch
    - 0011-Do-not-fetch-external-parameter-entities.patch
************************************************************************
IMPORTANT!  Nokogiri builds and uses a packaged version of libxml2.

If this is a concern for you and you want to use the system library
instead, abort this installation process and reinstall nokogiri as
follows:

    gem install nokogiri -- --use-system-libraries

If you are using Bundler, tell it to use the option:

    bundle config build.nokogiri --use-system-libraries
    bundle install

However, note that nokogiri does not necessarily support all versions
of libxml2.

For example, libxml2-2.9.0 and higher are currently known to be broken
and thus unsupported by nokogiri, due to compatibility problems and
XPath optimization bugs.
************************************************************************
Extracting libxml2-2.8.0.tar.gz into tmp/x86_64-apple-darwin13/ports/libxml2/2.8.0... OK
Running patch with /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/nokogiri-1.6.3.1/ports/patches/libxml2/0001-Fix-parser-local-buffers-size-problems.patch...
Running 'patch' for libxml2 2.8.0... OK
Running patch with /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/nokogiri-1.6.3.1/ports/patches/libxml2/0002-Fix-entities-local-buffers-size-problems.patch...
Running 'patch' for libxml2 2.8.0... OK
Running patch with /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/nokogiri-1.6.3.1/ports/patches/libxml2/0003-Fix-an-error-in-previous-commit.patch...
Running 'patch' for libxml2 2.8.0... OK
Running patch with /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/nokogiri-1.6.3.1/ports/patches/libxml2/0004-Fix-potential-out-of-bound-access.patch...
Running 'patch' for libxml2 2.8.0... OK
Running patch with /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/nokogiri-1.6.3.1/ports/patches/libxml2/0005-Detect-excessive-entities-expansion-upon-replacement.patch...
Running 'patch' for libxml2 2.8.0... OK
Running patch with /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/nokogiri-1.6.3.1/ports/patches/libxml2/0006-Do-not-fetch-external-parsed-entities.patch...
Running 'patch' for libxml2 2.8.0... OK
Running patch with /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/nokogiri-1.6.3.1/ports/patches/libxml2/0007-Enforce-XML_PARSER_EOF-state-handling-through-the-pa.patch...
Running 'patch' for libxml2 2.8.0... OK
Running patch with /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/nokogiri-1.6.3.1/ports/patches/libxml2/0008-Improve-handling-of-xmlStopParser.patch...
Running 'patch' for libxml2 2.8.0... OK
Running patch with /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/nokogiri-1.6.3.1/ports/patches/libxml2/0009-Fix-a-couple-of-return-without-value.patch...
Running 'patch' for libxml2 2.8.0... OK
Running patch with /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/nokogiri-1.6.3.1/ports/patches/libxml2/0010-Keep-non-significant-blanks-node-in-HTML-parser.patch...
Running 'patch' for libxml2 2.8.0... OK
Running patch with /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/nokogiri-1.6.3.1/ports/patches/libxml2/0011-Do-not-fetch-external-parameter-entities.patch...
Running 'patch' for libxml2 2.8.0... OK
Running 'configure' for libxml2 2.8.0... OK
Running 'compile' for libxml2 2.8.0... ERROR, review 'tmp/x86_64-apple-darwin13/ports/libxml2/2.8.0/compile.log' to see what happened.
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of necessary
libraries and/or headers.  Check the mkmf.log file for more details.  You may
need configuration options.

Provided configuration options:
    --with-opt-dir
    --without-opt-dir
    --with-opt-include
    --without-opt-include=${opt-dir}/include
    --with-opt-lib
    --without-opt-lib=${opt-dir}/lib
    --with-make-prog
    --without-make-prog
    --srcdir=.
    --curdir
    --ruby=/System/Library/Frameworks/Ruby.framework/Versions/2.0/usr/bin/ruby
    --help
    --clean
    --use-system-libraries
    --enable-static
    --disable-static
    --with-zlib-dir
    --without-zlib-dir
    --with-zlib-include
    --without-zlib-include=${zlib-dir}/include
    --with-zlib-lib
    --without-zlib-lib=${zlib-dir}/lib
    --enable-cross-build
    --disable-cross-build
/Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/mini_portile-0.6.0/lib/mini_portile.rb:279:in `block in execute': Failed to complete compile task (RuntimeError)
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/mini_portile-0.6.0/lib/mini_portile.rb:271:in `chdir'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/mini_portile-0.6.0/lib/mini_portile.rb:271:in `execute'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/mini_portile-0.6.0/lib/mini_portile.rb:70:in `compile'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/mini_portile-0.6.0/lib/mini_portile.rb:110:in `cook'
    from extconf.rb:253:in `block in process_recipe'
    from extconf.rb:154:in `tap'
    from extconf.rb:154:in `process_recipe'
    from extconf.rb:423:in `<main>'

Gem files will remain installed in /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/nokogiri-1.6.3.1 for inspection.
Results logged to /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/nokogiri-1.6.3.1/ext/nokogiri/gem_make.out
/Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers/unix_worker.rb:51:in `close': Interrupt
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers/unix_worker.rb:51:in `block (2 levels) in prepare_workers'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers/unix_worker.rb:35:in `fork'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers/unix_worker.rb:35:in `block in prepare_workers'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers/unix_worker.rb:31:in `times'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers/unix_worker.rb:31:in `each'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers/unix_worker.rb:31:in `map'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers/unix_worker.rb:31:in `prepare_workers'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers/worker.rb:20:in `initialize'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers/unix_worker.rb:20:in `initialize'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers.rb:14:in `new'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers.rb:14:in `worker_pool'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/installer.rb:280:in `install_in_parallel'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/installer.rb:88:in `run'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/installer.rb:18:in `install'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/cli/install.rb:78:in `run'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/cli.rb:145:in `install'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/vendor/thor/command.rb:27:in `run'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/vendor/thor/invocation.rb:121:in `invoke_command'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/vendor/thor.rb:363:in `dispatch'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/vendor/thor/base.rb:440:in `start'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/cli.rb:9:in `start'
    from /Users/chris/src/github/logstash/gembag.rb:68:in `block in <main>'
    from /Users/chris/src/github/logstash/gembag.rb:66:in `times'
    from /Users/chris/src/github/logstash/gembag.rb:66:in `<main>'
/Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/installer.rb:139:in `rescue in install_gem_from_spec': An error occurred while installing nokogiri (1.6.3.1), and Bundler cannot continue. (Bundler::InstallError)
Make sure that `gem install nokogiri -v '1.6.3.1'` succeeds before bundling.
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/installer.rb:99:in `install_gem_from_spec'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/installer.rb:282:in `block in install_in_parallel'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers/unix_worker.rb:42:in `call'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers/unix_worker.rb:42:in `block (2 levels) in prepare_workers'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers/unix_worker.rb:35:in `fork'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers/unix_worker.rb:35:in `block in prepare_workers'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers/unix_worker.rb:31:in `times'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers/unix_worker.rb:31:in `each'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers/unix_worker.rb:31:in `map'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers/unix_worker.rb:31:in `prepare_workers'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers/worker.rb:20:in `initialize'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers/unix_worker.rb:20:in `initialize'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers.rb:14:in `new'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/parallel_workers.rb:14:in `worker_pool'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/installer.rb:280:in `install_in_parallel'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/installer.rb:88:in `run'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/installer.rb:18:in `install'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/cli/install.rb:78:in `run'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/cli.rb:145:in `install'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/vendor/thor/command.rb:27:in `run'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/vendor/thor/invocation.rb:121:in `invoke_command'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/vendor/thor.rb:363:in `dispatch'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/vendor/thor/base.rb:440:in `start'
    from /Users/chris/src/github/logstash/vendor/bundle/ruby/2.0.0/gems/bundler-1.7.3/lib/bundler/cli.rb:9:in `start'
    from /Users/chris/src/github/logstash/gembag.rb:68:in `block in <main>'
    from /Users/chris/src/github/logstash/gembag.rb:66:in `times'
    from /Users/chris/src/github/logstash/gembag.rb:66:in `<main>'
chris@chris-macbook-pro ~/src/github/logstash (1.4.2)$ more tmp/x86_64-apple-darwin13/ports/libxml2/2.8.0/compile.log
tmp/x86_64-apple-darwin13/ports/libxml2/2.8.0/compile.log: No such file or directory
cdenneen commented 9 years ago

USE_JRUBY=1 bin/logstash deps works... can we update the README.md or is this going away in 1.5.x?

jordansissel commented 9 years ago

USE_JRUBY is gone as of June 13th in master and (as of right now) 1.x branches. Using JRuby is now the default (as you saw on master)

On master (4f6582168f687bf60d14c7e37fb4411c7b860bae) and 1.x (730363b2693d3d0a967078cdf96c5b7ba2dfac59), the README.md is has the correct instructions for those branches. On 1.4 branch and 1.4.2 tag, the USE_JRUBY=1 note is mentioned correctly.

The build steps for 1.4 are different than for 1.x/master branches.

cdenneen commented 9 years ago

Great thanks... I hadn't built in a while but something reminded me about USE_JRUBY after I walked away ;-)

nivedita2 commented 6 years ago

how to use nokogiri in logstash config file??