search

mesos / kafka

Apache Kafka on Apache Mesos
Apache License 2.0
414 stars 140 forks source link

Are there any prerequisites that were too obvious to mention in the readme? #195

Open d4goxn opened 8 years ago

d4goxn commented 8 years ago

I have just started exploring Kafka and Mesos. When I tried to run the first part of the install instructions:

# git clone https://github.com/mesos/kafka
# cd kafka
# ./gradlew jar

I get a few test failures at the ./gradlew jar. Most of the failed test mention a broker:

    BrokerTest. toJson_fromJson
    CliTest. broker_add
    CliTest. broker_list
    CliTest. broker_restart
    CliTest. broker_start_stop
    CliTest. broker_start_stop_timeout
    CliTest. broker_update
    ClusterTest. save_load
    ClusterTest. toJson_fromJson
    HttpServerTest. broker_add
    HttpServerTest. broker_add_range
    HttpServerTest. broker_list
    HttpServerTest. broker_start_stop
    HttpServerTest. broker_update
    SchedulerTest. newTask
    SchedulerTest. onFrameworkMessage
    SchedulerTest. sendReceiveBrokerLog

Do I need to already have a Mesos cluster running, or have Kafka available somewhere? Do I need to have some Scala dev tooling?

I'm going to poke around, see if I can get some insights from the test code itself, but this is also my first encounter with Scala, so I'm expecting to have the usual noob problems with a new stack, new language.

I'm on Ubuntu 16.04, with Java 1.8.

joestein commented 8 years ago

Hey @d4goxn any chance to reproduce in a vm setup? You don't need anything else except for what you are doing but you shouldn't be getting test failure. No mesos or kafka required. Your output looks odd.

git clone mesos/kafka cd src/vagrant vagrant up vagrant ssh cd /vagrant ./gradlew clean jar

compare to what you have local see or not

could be some other conflict local is causing issues, you can do a basic vm nothing init but java too to build e.g. https://github.com/elodina/scala-kafka

d4goxn commented 8 years ago

I get the same test failures in a VM, which was started from that Vagrantfile. The only deviation that I took from the process you posted was to specify which VM to SSH into: vagrant ssh master.

$ ./gradlew clean jar
:clean
:compileJava UP-TO-DATE
:compileScala
:processResources UP-TO-DATE
:classes
:compileTestJava UP-TO-DATE
:compileTestScala
[ant:scalac] Element '/vagrant/out/gradle/resources/main' does not exist.
:processTestResources UP-TO-DATE
:testClasses
:test

ly.stealth.mesos.kafka.HttpServerTest > broker_update FAILED
    java.io.IOException at HttpServerTest.scala:78
        Caused by: java.lang.NumberFormatException at HttpServerTest.scala:78

ly.stealth.mesos.kafka.HttpServerTest > broker_add_range FAILED
    java.io.IOException at HttpServerTest.scala:69
        Caused by: java.lang.NumberFormatException at HttpServerTest.scala:69

ly.stealth.mesos.kafka.HttpServerTest > broker_start_stop FAILED
    java.io.IOException at HttpServerTest.scala:166
        Caused by: java.lang.NumberFormatException at HttpServerTest.scala:166

ly.stealth.mesos.kafka.HttpServerTest > broker_list FAILED
    java.io.IOException at HttpServerTest.scala:129
        Caused by: java.lang.NumberFormatException at HttpServerTest.scala:129

ly.stealth.mesos.kafka.HttpServerTest > broker_add FAILED
    java.io.IOException at HttpServerTest.scala:51
        Caused by: java.lang.NumberFormatException at HttpServerTest.scala:51

ly.stealth.mesos.kafka.ClusterTest > save_load FAILED
    java.lang.NumberFormatException at ClusterTest.scala:64

ly.stealth.mesos.kafka.ClusterTest > toJson_fromJson FAILED
    java.lang.NumberFormatException at ClusterTest.scala:76

ly.stealth.mesos.kafka.CliTest > broker_update FAILED
    ly.stealth.mesos.kafka.Cli$Error at CliTest.scala:372

ly.stealth.mesos.kafka.CliTest > broker_start_stop_timeout FAILED
    java.lang.AssertionError at CliTest.scala:149

ly.stealth.mesos.kafka.CliTest > broker_start_stop FAILED
    ly.stealth.mesos.kafka.Cli$Error at CliTest.scala:372

ly.stealth.mesos.kafka.CliTest > broker_list FAILED
    ly.stealth.mesos.kafka.Cli$Error at CliTest.scala:372

ly.stealth.mesos.kafka.CliTest > broker_add FAILED
    ly.stealth.mesos.kafka.Cli$Error at CliTest.scala:372

ly.stealth.mesos.kafka.CliTest > broker_restart FAILED
    ly.stealth.mesos.kafka.Cli$Error at CliTest.scala:372

ly.stealth.mesos.kafka.BrokerTest > toJson_fromJson FAILED
    java.lang.NumberFormatException at BrokerTest.scala:356

ly.stealth.mesos.kafka.SchedulerTest > sendReceiveBrokerLog FAILED
    java.lang.AssertionError at SchedulerTest.scala:329

ly.stealth.mesos.kafka.SchedulerTest > onFrameworkMessage FAILED
    java.lang.AssertionError at SchedulerTest.scala:266

ly.stealth.mesos.kafka.SchedulerTest > newTask FAILED
    java.lang.NumberFormatException at SchedulerTest.scala:66

I wish I could be more helpful, looks like I need to get into Scala if I want to be able to work with this. Which is something I would have to do regardless, if I want to be able to troubleshoot issues with the Clojure wrapper that I was planning to use.

christianparpart commented 8 years ago

:+1: We're unfortunately running into the very same issue. Any chances to get a working documentation?

dmitrypekar commented 8 years ago

@d4goxn @christianparpart Can't reproduce this locally neither on 1.7.0_55-b13 nor on 1.8.0_60.

Could you please send a tarball of kafka-mesos/out/gradle/reports/tests/ folder? It contains test failure stacktraces, that would help us to clarify a reason why tests actually fail.

PS. You can also try to build using 1.7.