search

hobbit-project / platform

HOBBIT benchmarking platform
GNU General Public License v2.0
23 stars 9 forks source link

Challenge benchmarks do not get the correct RDF model #38

Closed MichaelRoeder closed 7 years ago

MichaelRoeder commented 7 years ago

Problem

When a "normal" experiment is started the benchmark controller of an example benchmark is created in the following way

2017-04-27 19:35:11,311 INFO [com.spotify.docker.client.DefaultDockerClient] - <Creating container with ContainerConfig: ContainerConfig{hostname=debsbenchmark-ec5d0b29b1e64fca9fdae4fa3294e85e, domainname=null, username=null, attachStdin=null, attachStdout=null, attachStderr=null, portSpecs=null, exposedPorts=null, tty=null, openStdin=null, stdinOnce=null, env=[HOBBIT_CONTAINER_NAME=debsbenchmark-ec5d0b29b1e64fca9fdae4fa3294e85e, constraint:org.hobbit.workergroup==benchmark, HOBBIT_RABBIT_HOST=rabbit, HOBBIT_SESSION_ID=1493321609589, HOBBIT_EXPERIMENT_URI=http://w3id.org/hobbit/experiments#1493321609589, BENCHMARK_PARAMETERS_MODEL={
  "@id" : "http://w3id.org/hobbit/experiments#New",
  "@type" : "http://w3id.org/hobbit/vocab#Experiment",
  "involvesBenchmark" : "http://project-hobbit.eu/resources/debs2017/debsbenchmark",
  "involvesSystemInstance" : "http://project-hobbit.eu/resources/debs2017/debsparrotsystemexample",
  "benchmarkMode" : "default",
  "dataPointCount" : "63",
  "format" : "0",
  "interval" : "10",
  "machineCount" : "1",
  "maxClusterIterations" : "50",
  "http://www.debs2017.org/gc/probabilityThreshold" : 0.005,
  "seed" : "123",
  "timeoutMinutes" : "-1",
  "transitionsCount" : "5",
  "windowSize" : "10",
  "@context" : {
    "windowSize" : {
      "@id" : "http://www.debs2017.org/gc/windowSize",
      "@type" : "http://www.w3.org/2001/XMLSchema#unsignedInt"
    },
    "timeoutMinutes" : {
      "@id" : "http://www.debs2017.org/gc/timeoutMinutes",
      "@type" : "http://www.w3.org/2001/XMLSchema#int"
    },
    "involvesBenchmark" : {
      "@id" : "http://w3id.org/hobbit/vocab#involvesBenchmark",
      "@type" : "@id"
    },
    "benchmarkMode" : {
      "@id" : "http://www.debs2017.org/gc/benchmarkMode"
    },
    "machineCount" : {
      "@id" : "http://www.debs2017.org/gc/machineCount",
      "@type" : "http://www.w3.org/2001/XMLSchema#unsignedInt"
    },
    "dataPointCount" : {
      "@id" : "http://www.debs2017.org/gc/dataPointCount",
      "@type" : "http://www.w3.org/2001/XMLSchema#unsignedInt"
    },
    "probabilityThreshold" : {
      "@id" : "http://www.debs2017.org/gc/probabilityThreshold",
      "@type" : "http://www.w3.org/2001/XMLSchema#double"
    },
    "format" : {
      "@id" : "http://www.debs2017.org/gc/format",
      "@type" : "http://www.w3.org/2001/XMLSchema#unsignedInt"
    },
    "seed" : {
      "@id" : "http://www.debs2017.org/gc/seed",
      "@type" : "http://www.w3.org/2001/XMLSchema#unsignedInt"
    },
    "interval" : {
      "@id" : "http://www.debs2017.org/gc/interval",
      "@type" : "http://www.w3.org/2001/XMLSchema#unsignedInt"
    },
    "involvesSystemInstance" : {
      "@id" : "http://w3id.org/hobbit/vocab#involvesSystemInstance",
      "@type" : "@id"
    },
    "transitionsCount" : {
      "@id" : "http://www.debs2017.org/gc/transitionsCount",
      "@type" : "http://www.w3.org/2001/XMLSchema#unsignedInt"
    },
    "maxClusterIterations" : {
      "@id" : "http://www.debs2017.org/gc/maxClusterIterations",
      "@type" : "http://www.w3.org/2001/XMLSchema#unsignedInt"
    }
  }
}
, HOBBIT_SYSTEM_URI=http://project-hobbit.eu/resources/debs2017/debsparrotsystemexample], cmd=null, image=git.project-hobbit.eu:4567/rkaterinenko/debsbenchmark, volumes=null, workingDir=null, entrypoint=null, networkDisabled=null, onBuild=null, labels={org.hobbit.type=benchmark}, macAddress=null, hostConfig=null, stopSignal=null}>

When creating a challenge task with the same (default) configuration as in the "normal" experiment the RDF model is not complete

2017-04-27 19:58:19,235 INFO [com.spotify.docker.client.DefaultDockerClient] - <Creating container with ContainerConfig: ContainerConfig{hostname=debsbenchmark-56795a7ea35141a79f0e9e9ba44ab009, domainname=null, username=null, attachStdin=null, attachStdout=null, attachStderr=null, portSpecs=null, exposedPorts=null, tty=null, openStdin=null, stdinOnce=null, env=[HOBBIT_CONTAINER_NAME=debsbenchmark-56795a7ea35141a79f0e9e9ba44ab009, constraint:org.hobbit.workergroup==benchmark, HOBBIT_RABBIT_HOST=rabbit, HOBBIT_SESSION_ID=1493323054466, HOBBIT_EXPERIMENT_URI=http://w3id.org/hobbit/experiments#1493323054466, BENCHMARK_PARAMETERS_MODEL={
  "@context" : {
    "hobbit" : "http://w3id.org/hobbit/vocab#",
    "ns5" : "http://project-hobbit.eu/resources/debs2017/",
    "rdf" : "http://www.w3.org/1999/02/22-rdf-syntax-ns#",
    "xsd" : "http://www.w3.org/2001/XMLSchema#",
    "rdfs" : "http://www.w3.org/2000/01/rdf-schema#",
    "ns2" : "http://w3id.org/hobbit/vocab#",
    "ns1" : "http://www.debs2017.org/gc/"
  }
}
, HOBBIT_SYSTEM_URI=http://project-hobbit.eu/resources/debs2017/debsparrotsystemnegative], cmd=null, image=git.project-hobbit.eu:4567/rkaterinenko/debsbenchmark, volumes=null, workingDir=null, entrypoint=null, networkDisabled=null, onBuild=null, labels={org.hobbit.type=benchmark}, macAddress=null, hostConfig=null, stopSignal=null}>
MichaelRoeder commented 7 years ago

Added the complete platform_controller_log.txt

MichaelRoeder commented 7 years ago

Notes for easier reproducing:

  1. Add the credentials of our admin to the compose file as described in https://github.com/hobbit-project/platform#using-benchmarks-from-the-hobbit-git

  2. We know that this issue has been reported for rkaterinenko. Create a user with this name and give it the role challenge-organiser as described at https://github.com/hobbit-project/platform#details-of-the-user-management

  3. For the experiment the DEBS GC challenge benchmark can be used.

yamalight commented 7 years ago

the problem was the fact that challenge model doesn't include graph, following snippet fixes it:

private Model createExpModelForChallengeTask(Model model, String challengeTaskUri, String systemUri) {
    Dataset dataset = DatasetFactory.create();
    dataset.addNamedModel("http://temp.org/challenge", model);
    String query = SparqlQueries.getCreateExperimentFromTaskQuery(Constants.NEW_EXPERIMENT_URI, challengeTaskUri,
            systemUri, "http://temp.org/challenge");
    if (query == null) {
        LOGGER.error("Couldn't load SPARQL query to create an RDF model for a new experiment. Returning null.");
        return null;
    }
    QueryExecution qe = QueryExecutionFactory.create(query, dataset);
    return qe.execConstruct();
}