search

hobbit-project / DEBS-GC-2018

Example of Sample System for DEBS Grand Challenge 2018
https://project-hobbit.eu/challenges/debs2018-grand-challenge/
0 stars 3 forks source link

SampleSystemTest::checkHealth sometimes not working / SystemAdapter code not executed #16

Closed florianschmidt1994 closed 6 years ago

florianschmidt1994 commented 6 years ago

Hey everyone,

I have a problem where the checkHealth test sometimes does not seem to execute the SystemAdapter code at all, but instead gets stuck in a loop of

2018-03-28 17:02:51,362 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:52,248 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:52,251 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>

A already put together a script to clean up all remnants of previous runs, but so far I was not able to see in which cases this works / in which cases it doesn't.

My cleanup steps

mvn clean package -DskipTests
docker ps -a | grep "java -cp" | awk '{ print $1 }' | xargs docker rm -f && docker rm -f rabbit
docker network rm hobbit-common
docker network rm hobbit

Please note that sometimes it does work (meaning I also see the accuracy score, logs, ...), it's just that I can't figure out what is necessary to have it work every time. I attached the full logs of that run below. Any help would be very much appreciated

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/florianschmidt/.m2/repository/org/hobbit/hobbit-java-sdk/1.1.1/hobbit-java-sdk-1.1.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/florianschmidt/.m2/repository/org/slf4j/slf4j-log4j12/1.7.15/slf4j-log4j12-1.7.15.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
2018-03-28 17:02:23,970 DEBUG [Jena] - <Jena initialization>
2018-03-28 17:02:26,938 DEBUG [RabbitMqDockerizer] - <Creating container (imageName=rabbitmq:latest)>
2018-03-28 17:02:27,228 DEBUG [RabbitMqDockerizer] - <Starting container (imageName=rabbitmq:latest, containerId=bd8548b843209a0ee833c9520dd1f687c64568b01e84d8ba0c1a150d43ba31b0)>
2018-03-28 17:02:27,667 DEBUG [RabbitMqDockerizer] - <Connecting container to networks (imageName=rabbitmq:latest)>
2018-03-28 17:02:28,436 DEBUG [RabbitMqDockerizer] - <Container started (imageName=rabbitmq:latest, containerId=bd8548b843209a0ee833c9520dd1f687c64568b01e84d8ba0c1a150d43ba31b0)>
2018-03-28 17:02:28,436 DEBUG [RabbitMqDockerizer] - <Starting monitoring & logs reading for container (imageName=rabbitmq:latest)>
2018-03-28 17:02:31,443 DEBUG [org.hobbit.sdk.docker.RabbitMqDockerizer] - <Trying to connect to container at rabbit (imageName=rabbitmq:latest)>
2018-03-28 17:02:37,643 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Initing CommandQueueListener>
2018-03-28 17:02:37,643 DEBUG [org.hobbit.sdk.utils.CommandQueueListener] - <Initializing...>
2018-03-28 17:02:37,714 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-03-28 17:02:37,714 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Running CommandQueueListener>
2018-03-28 17:02:37,714 DEBUG [org.hobbit.sdk.utils.CommandQueueListener] - <Initialized. Waiting for termination signal>
2018-03-28 17:02:37,716 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Initing benchmark-controller-dockerizer>
2018-03-28 17:02:37,716 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Running benchmark-controller-dockerizer>
2018-03-28 17:02:37,716 DEBUG [benchmark-controller-dockerizer] - <Removing containers (imageName=git.project-hobbit.eu:4567/debs_2018_gc/benchmark-controller)>
2018-03-28 17:02:37,716 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Initing SystemAdapter>
2018-03-28 17:02:37,804 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-03-28 17:02:37,816 DEBUG [benchmark-controller-dockerizer] - <Creating container (imageName=git.project-hobbit.eu:4567/debs_2018_gc/benchmark-controller)>
2018-03-28 17:02:37,874 DEBUG [org.hobbit.debs_2018_gc_samples.System.SystemAdapter] - <Init()>
2018-03-28 17:02:37,878 DEBUG [org.hobbit.debs_2018_gc_samples.System.SystemAdapter] - <SystemModel: {
  "@id" : "http://jenaKeyValue.com/experimentUri",
  "queryType" : "2",
  "@context" : {
    "queryType" : {
      "@id" : "http://project-hobbit.eu/sml-benchmark-v2/queryType",
      "@type" : "http://www.w3.org/2001/XMLSchema#int"
    }
  }
}
>
2018-03-28 17:02:37,878 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Running SystemAdapter>
2018-03-28 17:02:37,939 DEBUG [benchmark-controller-dockerizer] - <Starting container (imageName=git.project-hobbit.eu:4567/debs_2018_gc/benchmark-controller, containerId=720c72ca57c6cfb452f1ce662e4192ada69d0be7069eb60848301dba13aab104)>
2018-03-28 17:02:38,329 DEBUG [benchmark-controller-dockerizer] - <Connecting container to networks (imageName=git.project-hobbit.eu:4567/debs_2018_gc/benchmark-controller)>
2018-03-28 17:02:38,685 DEBUG [benchmark-controller-dockerizer] - <Container started (imageName=git.project-hobbit.eu:4567/debs_2018_gc/benchmark-controller, containerId=720c72ca57c6cfb452f1ce662e4192ada69d0be7069eb60848301dba13aab104)>
2018-03-28 17:02:38,685 DEBUG [benchmark-controller-dockerizer] - <Starting monitoring & logs reading for container (imageName=git.project-hobbit.eu:4567/debs_2018_gc/benchmark-controller)>
2018-03-28 17:02:39,948 DEBUG [benchmark-controller-dockerizer] - <2018-03-28 15:02:40,005 DEBUG [Jena] - <Jena initialization>
>
2018-03-28 17:02:40,366 DEBUG [org.hobbit.sdk.utils.commandreactions.MultipleCommandsReaction] - <CONTAINER_START signal received with imageName=git.project-hobbit.eu:4567/debs_2018_gc/data-generator>
2018-03-28 17:02:40,367 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Initing data-generator-dockerizer>
2018-03-28 17:02:40,367 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Running data-generator-dockerizer>
2018-03-28 17:02:40,367 DEBUG [data-generator-dockerizer] - <Removing containers (imageName=git.project-hobbit.eu:4567/debs_2018_gc/data-generator)>
2018-03-28 17:02:40,441 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-03-28 17:02:40,463 DEBUG [data-generator-dockerizer] - <Creating container (imageName=git.project-hobbit.eu:4567/debs_2018_gc/data-generator)>
2018-03-28 17:02:40,473 DEBUG [org.hobbit.sdk.utils.commandreactions.MultipleCommandsReaction] - <CONTAINER_START signal received with imageName=git.project-hobbit.eu:4567/debs_2018_gc/task-generator>
2018-03-28 17:02:40,475 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Initing TaskGenerator>
2018-03-28 17:02:40,540 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-03-28 17:02:40,542 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-03-28 17:02:40,572 DEBUG [data-generator-dockerizer] - <Starting container (imageName=git.project-hobbit.eu:4567/debs_2018_gc/data-generator, containerId=b7c9cb815a5ff877cfdf67c23758d11165ce907f240d64b3f63884940274d7ae)>
2018-03-28 17:02:40,603 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <Init()>
2018-03-28 17:02:40,605 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <GENERATOR_LIMIT=72>
2018-03-28 17:02:40,605 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <GENERATOR_TIMEOUT=60>
2018-03-28 17:02:40,605 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <Init (queryType=0, recordsLimit=72, timeout=60)>
2018-03-28 17:02:40,639 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <Reading data/debs2018_training_labeled.csv>
2018-03-28 17:02:40,993 DEBUG [benchmark-controller-dockerizer] - <2018-03-28 15:02:41,016 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <Init()>
2018-03-28 15:02:41,068 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <BenchmarkModel: {
  "@id" : "http://jenaKeyValue.com/experimentUri",
  "generatorLimit" : "72",
  "generatorTimeoutMin" : "60",
  "queryType" : "2",
  "@context" : {
    "generatorLimit" : {
      "@id" : "http://project-hobbit.eu/sml-benchmark-v2/generatorLimit",
      "@type" : "http://www.w3.org/2001/XMLSchema#int"
    },
    "generatorTimeoutMin" : {
      "@id" : "http://project-hobbit.eu/sml-benchmark-v2/generatorTimeoutMin",
      "@type" : "http://www.w3.org/2001/XMLSchema#int"
    },
    "queryType" : {
      "@id" : "http://project-hobbit.eu/sml-benchmark-v2/queryType",
      "@type" : "http://www.w3.org/2001/XMLSchema#int"
    }
  }
}
>
2018-03-28 15:02:41,069 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <Gen.EnvVariables: QUERY_TYPE=2, GENERATOR_LIMIT=72, GENERATOR_TIMEOUT=60>
2018-03-28 15:02:41,069 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <createDataGenerators()>
2018-03-28 15:02:41,183 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <createTaskGenerators()>
2018-03-28 15:02:41,278 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <waitForComponents()>
2018-03-28 15:02:41,279 DEBUG [org.hobbit.core.components.AbstractBenchmarkController] - <Received EVAL_STORAGE_READY_SIGNAL>
>
2018-03-28 17:02:41,524 DEBUG [data-generator-dockerizer] - <Connecting container to networks (imageName=git.project-hobbit.eu:4567/debs_2018_gc/data-generator)>
2018-03-28 17:02:41,537 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <File reading finished>
2018-03-28 17:02:41,538 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <Processing 73 lines>
2018-03-28 17:02:41,547 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <Processing finished: 72 tuples, 2 trips>
2018-03-28 17:02:41,553 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Running TaskGenerator>
2018-03-28 17:02:41,928 DEBUG [data-generator-dockerizer] - <Container started (imageName=git.project-hobbit.eu:4567/debs_2018_gc/data-generator, containerId=b7c9cb815a5ff877cfdf67c23758d11165ce907f240d64b3f63884940274d7ae)>
2018-03-28 17:02:41,928 DEBUG [data-generator-dockerizer] - <Starting monitoring & logs reading for container (imageName=git.project-hobbit.eu:4567/debs_2018_gc/data-generator)>
2018-03-28 17:02:42,012 DEBUG [benchmark-controller-dockerizer] - <2018-03-28 15:02:42,293 DEBUG [org.hobbit.core.components.AbstractBenchmarkController] - <Received TASK_GENERATOR_READY_SIGNAL>
>
2018-03-28 17:02:42,089 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-03-28 17:02:42,104 DEBUG [org.hobbit.sdk.utils.commandreactions.MultipleCommandsReaction] - <CONTAINER_START signal received with imageName=git.project-hobbit.eu:4567/debs_2018_gc/eval-storage>
2018-03-28 17:02:42,105 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Initing eval-storage-dockerizer>
2018-03-28 17:02:42,106 DEBUG [org.hobbit.sdk.ComponentsExecutor] - <Running eval-storage-dockerizer>
2018-03-28 17:02:42,106 DEBUG [eval-storage-dockerizer] - <Removing containers (imageName=git.project-hobbit.eu:4567/debs_2018_gc/eval-storage)>
2018-03-28 17:02:42,283 DEBUG [eval-storage-dockerizer] - <Creating container (imageName=git.project-hobbit.eu:4567/debs_2018_gc/eval-storage)>
2018-03-28 17:02:42,284 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-03-28 17:02:42,291 INFO [org.hobbit.core.components.AbstractTaskGenerator] - <Received signal to start.>
2018-03-28 17:02:42,340 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <generateTask()>
2018-03-28 17:02:42,341 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <Start sending tuples(2 ships)>
2018-03-28 17:02:42,375 DEBUG [eval-storage-dockerizer] - <Starting container (imageName=git.project-hobbit.eu:4567/debs_2018_gc/eval-storage, containerId=f76d6a74a351189bf6102f9f4e9e704c09f49068733c60e1e0884123d092a3b0)>
2018-03-28 17:02:42,753 DEBUG [eval-storage-dockerizer] - <Connecting container to networks (imageName=git.project-hobbit.eu:4567/debs_2018_gc/eval-storage)>
2018-03-28 17:02:43,026 DEBUG [benchmark-controller-dockerizer] - <2018-03-28 15:02:42,773 DEBUG [org.hobbit.core.components.AbstractBenchmarkController] - <Received DATA_GENERATOR_READY_SIGNAL>
2018-03-28 15:02:42,826 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <createEvaluationStorage()>
2018-03-28 15:02:43,023 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <executeBenchmark(sending TASK_GENERATOR_START_SIGNAL & DATA_GENERATOR_START_SIGNAL)>
2018-03-28 15:02:43,023 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <waitForDataGenToFinish() to send DATA_GENERATION_FINISHED_SIGNAL>
2018-03-28 15:02:43,025 DEBUG [org.hobbit.core.components.AbstractBenchmarkController] - <Waiting for 1 Data Generators to finish.>
>
2018-03-28 17:02:43,047 DEBUG [eval-storage-dockerizer] - <Container started (imageName=git.project-hobbit.eu:4567/debs_2018_gc/eval-storage, containerId=f76d6a74a351189bf6102f9f4e9e704c09f49068733c60e1e0884123d092a3b0)>
2018-03-28 17:02:43,047 DEBUG [eval-storage-dockerizer] - <Starting monitoring & logs reading for container (imageName=git.project-hobbit.eu:4567/debs_2018_gc/eval-storage)>
2018-03-28 17:02:43,344 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 2 tuples/s>
2018-03-28 17:02:44,048 DEBUG [benchmark-controller-dockerizer] - <2018-03-28 15:02:44,243 DEBUG [org.hobbit.core.components.AbstractBenchmarkController] - <Received EVAL_STORAGE_READY_SIGNAL>
>
2018-03-28 17:02:44,093 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:44,207 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Init()>
2018-03-28 15:02:44,235 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Init (queryType=2, experimentUri=http://example.com/exp1, systemContainerId=git.project-hobbit.eu:4567/smirnp/sml-benchmark-v2/system-adapter>
>
2018-03-28 17:02:44,348 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:45,114 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:45,241 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (2 t/s) Actuals: 2 (2 t/s)>
>
2018-03-28 17:02:45,348 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:45,607 DEBUG [data-generator-dockerizer] - <2018-03-28 15:02:45,300 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.generator.DataGenerator] - <close()>
>
2018-03-28 17:02:46,136 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:46,241 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>
>
2018-03-28 17:02:46,353 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:46,664 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-03-28 17:02:46,693 INFO [org.hobbit.core.components.AbstractTaskGenerator] - <Received signal to finish.>
2018-03-28 17:02:46,750 INFO [org.hobbit.core.components.AbstractCommandReceivingComponent] - <Couldn't get the id of this Docker container. Won't be able to create containers.>
2018-03-28 17:02:46,751 INFO [org.hobbit.core.components.AbstractTaskGenerator] - <Received signal to finish.>
2018-03-28 17:02:47,106 DEBUG [benchmark-controller-dockerizer] - <2018-03-28 15:02:47,414 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.BenchmarkController] - <waitForTaskGenToFinish() to finish to send TASK_GENERATION_FINISHED_SIGNAL>
2018-03-28 15:02:47,415 DEBUG [org.hobbit.core.components.AbstractBenchmarkController] - <Waiting for 1 Task Generators to finish.>
>
2018-03-28 17:02:47,158 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:47,244 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>
>
2018-03-28 17:02:47,355 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:48,174 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:48,245 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>
>
2018-03-28 17:02:48,352 DEBUG [org.hobbit.core.rabbit.DataReceiverImpl] - <Receiver task terminates after receiving 1 messages.>
2018-03-28 17:02:48,359 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:49,198 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:49,246 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>
>
2018-03-28 17:02:49,360 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:50,214 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:50,248 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>
>
2018-03-28 17:02:50,361 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:51,230 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:51,250 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>
>
2018-03-28 17:02:51,362 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:52,248 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:52,251 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>
>
2018-03-28 17:02:52,367 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:53,277 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:53,254 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>
>
2018-03-28 17:02:53,371 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:54,297 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:54,255 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>
>
2018-03-28 17:02:54,376 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
2018-03-28 17:02:55,317 DEBUG [eval-storage-dockerizer] - <2018-03-28 15:02:55,255 DEBUG [org.hobbit.smlbenchmark_v2.benchmark.EvalStorage] - <Expectations: 2 (0 t/s) Actuals: 2 (0 t/s)>
>
2018-03-28 17:02:55,378 DEBUG [org.hobbit.debs_2018_gc_samples.Benchmark.TaskGenerator] - <tuples 2 sent. Curr: 0 tuples/s>
Disconnected from the target VM, address: '127.0.0.1:61273', transport: 'socket'

Process finished with exit code 137 (interrupted by signal 9: SIGKILL)
florianschmidt1994 commented 6 years ago

Update: Currently it looks like after cleaning my setup and rebuilding the jar, the local benchmark works only from the second time on that it is run, but consistently afterwards.

smirnp commented 6 years ago

Hi! Can you please confirm, that you do not have a local installation of the HOBBIT platform? SDK conficts with it when it is running. If no platform is running, then the problem can be because of rabbitMQ may contain old queues from the previous run (by default it is not stopped/killed by the SDK after execution). To check that you may check whether no rabbit container is running when you start the checkHealth(). If it will help, then you may set .useCachedContainer(false) for the rabbitMqDockerizer in test file to force its killing before every run.

P.s.: under which OS and Java version you are running?

florianschmidt1994 commented 6 years ago

I built and installed a local version of the java sdk with mvn clean install, is that not the way to do it?

When running it with useCachedImages = false it never succeeds, for some reason it seems like something is not initialized correctly during the first run.

I'm runnning MacOS 10.13.3 with jdk1.8.0_131

smirnp commented 6 years ago

Seems, I reproduced the problem. Trying to find a solution.

smirnp commented 6 years ago

I have updated the SDK to 1.1.4 (packaged jar is under lib folder, you may install in by doing the mvn validate). I also updated images for benchmark-controller and eval-storage. Delete please the old ones on your machine by docker rmi <imageName>, the new-one will be pulled by the SDK automatically.

I found the problem of not finishing execution only if multiple benchmark-controller containers were executed at the moment (run two executions in parallel, or old containers were not killed then I interrupted execution manually). This is not directly your problem, but probably updates solution should help. If not, then provide please full logs again, now they should contain some additional information for debugging.

smirnp commented 6 years ago

Recently I have fixed some other bug with TaskGenerator. Please try the updated code from the repository.

florianschmidt1994 commented 6 years ago

It now seems to be working, thanks!