search

ondra-m / ruby-spark

Ruby wrapper for Apache Spark
MIT License
227 stars 29 forks source link

Class org.apache.spark.SparkConf is missing. Make sure that Spark and RubySpark is assembled. (Spark::JavaBridgeError) #9

Closed ruanwz closed 9 years ago

ruanwz commented 9 years ago

I followed the installation steps, but got the following error when running tests. Is there anything I have missed? Thanks!

git clone ...
bundle install
rake compile
rake
bin/ruby-spark build
rake
/Users/david/.rvm/rubies/ruby-2.2.1/bin/ruby -I/Users/david/.rvm/gems/ruby-2.2.1/gems/rspec-core-3.3.1/lib:/Users/david/.rvm/gems/ruby-2.2.1/gems/rspec-support-3.3.0/lib /Users/david/.rvm/gems/ruby-2.2.1/gems/rspec-core-3.3.1/exe/rspec --pattern spec/\*\*\{,/\*/\*\*\}/\*_spec.rb
Coverage report generated for RSpec to /Users/david/gitrepos/ruby-spark/coverage. 316 / 458 LOC (69.0%) covered.
/Users/david/gitrepos/ruby-spark/lib/spark/java_bridge/base.rb:198:in `raise_missing_class': Class org.apache.spark.SparkConf is missing. Make sure that Spark and RubySpark is assembled. (Spark::JavaBridgeError)
    from /Users/david/gitrepos/ruby-spark/lib/spark/java_bridge/rjb.rb:20:in `rescue in import'
    from /Users/david/gitrepos/ruby-spark/lib/spark/java_bridge/rjb.rb:18:in `import'
    from /Users/david/gitrepos/ruby-spark/lib/spark/java_bridge/base.rb:53:in `block in import_all'
    from /Users/david/gitrepos/ruby-spark/lib/spark/java_bridge/base.rb:52:in `each'
    from /Users/david/gitrepos/ruby-spark/lib/spark/java_bridge/base.rb:52:in `import_all'
    from /Users/david/gitrepos/ruby-spark/lib/spark.rb:214:in `load_lib'
    from /Users/david/gitrepos/ruby-spark/spec/spec_helper.rb:9:in `<top (required)>'
    from /Users/david/gitrepos/ruby-spark/spec/lib/collect_spec.rb:1:in `require'
    from /Users/david/gitrepos/ruby-spark/spec/lib/collect_spec.rb:1:in `<top (required)>'
    from /Users/david/.rvm/gems/ruby-2.2.1/gems/rspec-core-3.3.1/lib/rspec/core/configuration.rb:1327:in `load'
    from /Users/david/.rvm/gems/ruby-2.2.1/gems/rspec-core-3.3.1/lib/rspec/core/configuration.rb:1327:in `block in load_spec_files'
    from /Users/david/.rvm/gems/ruby-2.2.1/gems/rspec-core-3.3.1/lib/rspec/core/configuration.rb:1325:in `each'
    from /Users/david/.rvm/gems/ruby-2.2.1/gems/rspec-core-3.3.1/lib/rspec/core/configuration.rb:1325:in `load_spec_files'
    from /Users/david/.rvm/gems/ruby-2.2.1/gems/rspec-core-3.3.1/lib/rspec/core/runner.rb:102:in `setup'
    from /Users/david/.rvm/gems/ruby-2.2.1/gems/rspec-core-3.3.1/lib/rspec/core/runner.rb:88:in `run'
    from /Users/david/.rvm/gems/ruby-2.2.1/gems/rspec-core-3.3.1/lib/rspec/core/runner.rb:73:in `run'
    from /Users/david/.rvm/gems/ruby-2.2.1/gems/rspec-core-3.3.1/lib/rspec/core/runner.rb:41:in `invoke'
    from /Users/david/.rvm/gems/ruby-2.2.1/gems/rspec-core-3.3.1/exe/rspec:4:in `<main>'
/Users/david/.rvm/rubies/ruby-2.2.1/bin/ruby -I/Users/david/.rvm/gems/ruby-2.2.1/gems/rspec-core-3.3.1/lib:/Users/david/.rvm/gems/ruby-2.2.1/gems/rspec-support-3.3.0/lib /Users/david/.rvm/gems/ruby-2.2.1/gems/rspec-core-3.3.1/exe/rspec --pattern spec/\*\*\{,/\*/\*\*\}/\*_spec.rb failed
ondra-m commented 9 years ago

What about bin/ruby-spark shell?

If there will be the same problem please paste output from bin/ruby-spark build.

ruanwz commented 9 years ago

@ondra-m When trying ruby-spark before, I didn't installed spark, I only use ruby-spark build. Now I install spark again from http://spark.apache.org/ and try again, now it is working.