search

mesos / chronos

Fault tolerant job scheduler for Mesos which handles dependencies and ISO8601 based schedules
http://mesos.github.io/chronos/
Apache License 2.0
4.39k stars 529 forks source link

Build Failure on Ubuntu 12.04 64bit #28

Closed zangzhw closed 11 years ago

zangzhw commented 11 years ago

I tried to install the chronos on Ubuntu 12.04 which has been updated. I searched on Google but the result doesn't help. So I just wonder if there is a bug or I have done something wrong.

Here is part of the output

Tests run: 2, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 0.128 sec <<< FAILURE!
JobScheduler should::A job creates a failed task and then a successful task from a synchronous job(com.airbnb.scheduler.jobs.JobSchedulerIntegrationTest)  Time elapsed: 0.039 sec  <<< FAILURE!
org.specs2.reporter.JUnitReporter$$anon$1: '0' is not equal to '1' expected:<[1]> but was:<[0]>
    at com.airbnb.scheduler.jobs.JobSchedulerIntegrationTest$$anonfun$1$$anonfun$apply$11.apply(JobSchedulerIntegrationTest.scala:31)
    at com.airbnb.scheduler.jobs.JobSchedulerIntegrationTest$$anonfun$1$$anonfun$apply$11.apply(JobSchedulerIntegrationTest.scala:14)

Running com.airbnb.scheduler.jobs.Iso8601ExpressionParserSpec
Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.04 sec
Running com.airbnb.scheduler.state.PersistenceStoreSpec
Apr 23, 2013 9:45:15 PM com.airbnb.scheduler.state.MesosStatePersistenceStore persistJob
INFO: Persisting job 'sample-name' with data 'ScheduleBasedJob(R1/2012-01-01T00:00:01.000Z/PT1M,sample-name,sample-command,PT1H,1,0,fooexecutor,args,2,,,,false)'
Apr 23, 2013 9:45:15 PM com.airbnb.scheduler.state.MesosStatePersistenceStore persistData
INFO: State J_sample-name does not exist yet. Adding to state
Apr 23, 2013 9:45:15 PM com.airbnb.scheduler.state.MesosStatePersistenceStore persistData
INFO: State update successful: true
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.039 sec
Running com.airbnb.scheduler.api.SerDeTest
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.052 sec

Results :

Failed tests: 
   '0' is not equal to '1' expected:<[1]> but was:<[0]>

Tests run: 35, Failures: 1, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 21.220s
[INFO] Finished at: Tue Apr 23 21:45:15 PDT 2013
[INFO] Final Memory: 17M/66M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.13:test (default-test) on project chronos: There are test failures.
[ERROR] 
[ERROR] Please refer to /usr/local/chronos/target/surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

I found this post https://github.com/airbnb/chronos/issues/21 and tried to run mvn clean and then mvn package, but still failed when I run mvn package

the maven and java I used is as following:

# java -version
java version "1.7.0_15"
OpenJDK Runtime Environment (IcedTea7 2.3.7) (7u15-2.3.7-0ubuntu1~12.04.1)
OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)
# javac -version
javac 1.7.0_15
# mvn -version
Apache Maven 3.0.4
Maven home: /usr/share/maven
Java version: 1.7.0_15, vendor: Oracle Corporation
Java home: /usr/lib/jvm/java-7-openjdk-amd64/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "linux", version: "3.5.0-27-generic", arch: "amd64", family: "unix"

Thank you!

zangzhw commented 11 years ago

I skip the test for surefire and it seems OK now.

ronaldgames commented 11 years ago

how you did the skip?

florianleibert commented 11 years ago

-Dmaven.test.skip=true

zangzhw commented 11 years ago

http://maven.apache.org/surefire/maven-surefire-plugin/examples/skipping-test.html

<skipTests>true</skipTests>

utsengar commented 11 years ago

+1 This is still a bug. Adding: -DskipTests=true after mvn package bypasses it.

zangzhw commented 11 years ago

Update.

I skipped test and successfully install the chronos. It seems OK, but when I try to create a test job, such as echo "foo" >> /tmp/chronos_test or date > /temp/test_date,both "#ERROR" and "SUCCESS" are 0 on WebUI. It seems that the job is not executed, while I tried to "force run" the task, it's okay, and the last success shows the correct time. I tried to change the logging level in local_scheduler_nozk.yml to ALL, there was some folder generated in /tmp/mesos/slaves/, like /tmp/mesos/slaves/<...>/frameworks/<...>/executors/default/<...>/runs/<...>/, but there were no log file in these folder. I tried latest mesos 0.12.x and the revision chronos install script provided, neither works. Is this a bug of chronos or mesos?

Need Help. Thank you.

andykram commented 11 years ago

@zangzhw is this still an issue with a fresh pull? Additionally, could you please provide the output from /usr/local/chronos/target/surefire-reports

zangzhw commented 11 years ago

Hi Andrew @andykram, I successfully finished installation on my laptop instead of the VM (both run Ubuntu 12.04 and updated) after a fresh pull several days ago. But there is still error in the VM environment. Here are the output on the VM. It shown Could not find artifact org.apache.mesos:mesos:jar:0.12.0-SNAPSHOT. I checked /root/.m2/repository/org/apache/mesos/mesos/, the 0.12.0-SNAPSHOT did exist. 

# ./bin/installer.bash 

Welcome to the interactive chronos installation. This script will first install mesos and then chronos.

Depending on your hardware and internet connection, this script might take 15 - 25 minutes as we're compiling mesos from source.
If you run into any issues, please check the FAQ in the chronos repo.
Do you have mesos installed already? Type 'yes' or 'no' followed by [ENTER]:
yes
Type the path of the compiled mesos installation followed by [ENTER]:
/usr/local/mesos
Skipping mesos installation
/usr/local/mesos /usr/local/chronos
Installing snapshot of mesos version 0.12.0 into local mvn repository
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- maven-install-plugin:2.3:install-file (default-cli) @ standalone-pom ---
[INFO] Installing /usr/local/mesos/src/build/src/mesos-0.12.0.jar
./src/build12/src/mesos-0.12.0.jar to /root/.m2/repository/org/apache/mesos/mesos/0.12.0-SNAPSHOT/mesos-0.12.0-SNAPSHOT.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 1.135s
[INFO] Finished at: Mon May 20 17:35:45 PDT 2013
[INFO] Final Memory: 5M/30M
[INFO] ------------------------------------------------------------------------
Replacing pom.xml mesos dependency
/usr/local/chronos
/usr/local/chronos/bin /usr/local/chronos
Installing chronos
[INFO] Scanning for projects...
[WARNING] 
[WARNING] Some problems were encountered while building the effective model for com.airbnb:chronos:jar:1.0-SNAPSHOT
[WARNING] 'build.plugins.plugin.(groupId:artifactId)' must be unique but found duplicate declaration of plugin net.alchim31.maven:scala-maven-plugin @ line 369, column 21
[WARNING] 
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING] 
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING] 
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building chronos 1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 3.352s
[INFO] Finished at: Mon May 20 17:35:50 PDT 2013
[INFO] Final Memory: 9M/30M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project chronos: Could not resolve dependencies for project com.airbnb:chronos:jar:1.0-SNAPSHOT: Could not find artifact org.apache.mesos:mesos:jar:0.12.0-SNAPSHOT -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException

# ls /root/.m2/repository/org/apache/mesos/mesos/
0.12.0-SNAPSHOT  maven-metadata-local.xml

On other other hand, chronos on my laptop was installed but has some problem when running as I mentioned in previous comment, the scheduled task doesn't success while the force run is okay.

zangzhw commented 11 years ago

@andykram I just did a fresh pull on my laptop and re-install the chronos it seems ok on my laptop, while still failed on my VM.

rgahagan commented 11 years ago

I've not been having problems in Vagrant with a 12.04 image, but on our cloud server I am seeing this problem.

-DskipTests=true does allow this to work on the AWS image.

@andykram here's the dir listing for /usr/local/chronos/target/surefire-reports. not sure which one's pertinent.

  com.airbnb.scheduler.api.SerDeTest.txt
  com.airbnb.scheduler.jobs.graph.JobGraphSpec.txt
  com.airbnb.scheduler.jobs.Iso8601ExpressionParserSpec.txt
  com.airbnb.scheduler.jobs.JobSchedulerIntegrationTest.txt
  com.airbnb.scheduler.jobs.JobSchedulerSpec.txt
  com.airbnb.scheduler.jobs.JobUtilsSpec.txt
  com.airbnb.scheduler.jobs.ScheduleStreamSpec.txt
  com.airbnb.scheduler.jobs.TaskManagerSpec.txt
  com.airbnb.scheduler.state.PersistenceStoreSpec.txt
  TEST-com.airbnb.scheduler.api.SerDeTest.xml
  TEST-com.airbnb.scheduler.jobs.graph.JobGraphSpec.xml
  TEST-com.airbnb.scheduler.jobs.Iso8601ExpressionParserSpec.xml
  TEST-com.airbnb.scheduler.jobs.JobSchedulerIntegrationTest.xml
  TEST-com.airbnb.scheduler.jobs.JobSchedulerSpec.xml
  TEST-com.airbnb.scheduler.jobs.JobUtilsSpec.xml
  TEST-com.airbnb.scheduler.jobs.ScheduleStreamSpec.xml
  TEST-com.airbnb.scheduler.jobs.TaskManagerSpec.xml
  TEST-com.airbnb.scheduler.state.PersistenceStoreSpec.xml
deanchen commented 11 years ago

Which version of java are you using? The test failures went away after I downgraded the java jre/jdk from 1.7 to 1.6.

zangzhw commented 11 years ago

@deanchen Thank you. I used java 1.7. I did try 1.6 before but there were other errors. I didn't try 1.6 with latest chronos code, I will try it later.

rgahagan commented 11 years ago

I'm running 1.7 as 1.6 introduced a Scala compile error

java -version java version "1.7.0_21" OpenJDK Runtime Environment (IcedTea 2.3.9) (7u21-2.3.9-0ubuntu0.12.04.1) OpenJDK 64-Bit Server VM (build 23.7-b01, mixed mode)

florianleibert commented 11 years ago

This is a bug of OpenJDK 1.6. Closing the issue!