search

brianfrankcooper / YCSB

Yahoo! Cloud Serving Benchmark
Apache License 2.0
4.96k stars 2.25k forks source link

Run “mvn clean package” command and the reactor build order has no "mapkeeper binding" #885

Closed CaixinGong closed 7 years ago

CaixinGong commented 7 years ago

I followed https://github.com/m1ch1/mapkeeper/wiki/Getting-Started to install mapkeeper to test LevelDB, but when I run “mvn clean package” command in YCSB directory, I get the following outputs which has no "mapkeeper binding":

[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for com.yahoo.ycsb:hbase094-binding:jar:0.13.0-SNAPSHOT
[WARNING] 'dependencies.dependency.exclusions.exclusion.groupId' for com.yahoo.ycsb:hbase098-binding:jar with value '' does not match a valid id pattern. @ line 51, column 20
[WARNING] 'dependencies.dependency.exclusions.exclusion.artifactId' for com.yahoo.ycsb:hbase098-binding:jar with value '' does not match a valid id pattern. @ line 52, column 23
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO]
[INFO] YCSB Root
[INFO] Core YCSB
[INFO] Per Datastore Binding descriptor
[INFO] YCSB Datastore Binding Parent
[INFO] Accumulo DB Binding
[INFO] Aerospike DB Binding
[INFO] ArangoDB Binding
[INFO] AsyncHBase Client Binding for Apache HBase
[INFO] Azure table storage Binding
[INFO] Cassandra 2.1+ DB Binding
[INFO] Couchbase Binding
[INFO] Couchbase Java SDK 2.x Binding
[INFO] Azure DocumentDB Binding
[INFO] DynamoDB DB Binding
[INFO] Elasticsearch Binding
[INFO] Geode DB Binding
[INFO] Google Cloud Datastore Binding
[INFO] Google Cloud Bigtable Binding
[INFO] HBase 0.98.x DB Binding
[INFO] HBase 0.94.x DB Binding
[INFO] HBase 1.0 DB Binding
[INFO] Hypertable DB Binding
[INFO] Infinispan DB Binding
[INFO] JDBC DB Binding
[INFO] Kudu DB Binding
[INFO] memcached binding
[INFO] MongoDB Binding
[INFO] Oracle NoSQL Database Binding
[INFO] OrientDB Binding
[INFO] rados of Ceph FS binding
[INFO] Redis DB Binding
[INFO] Rest Client Binding
[INFO] Riak KV Binding
[INFO] S3 Storage Binding
[INFO] Solr Binding
[INFO] Solr 6 Binding
[INFO] Tarantool DB Binding
[INFO] YCSB Release Distribution Builder
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building YCSB Root 0.13.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------

I don't know why it happens and how to solve it. It seems the the content of Get Started(https://github.com/m1ch1/mapkeeper/wiki/Getting-Started) is too old, could anyone update it? I can not successfully install the mapkeeper binding.

risdenk commented 7 years ago

The getting started you referenced (https://github.com/m1ch1/mapkeeper/wiki/Getting-Started) is not from this project. For this project see: #181 and #262

CaixinGong commented 7 years ago

I followed the way described in #262(The method of Compiling mapkeeper temporarily). I redescribe the steps I do: step 1: run mvn clean package in YCSB(the latest version 0.12.0) directory After step 1, I can run./bin/ycsb load basic workloads/workloada succcessly. step 2: follow "https://github.com/m1ch1/mapkeeper/wiki/Getting-Started" to Generate Thrift MapKeeper Binding a. install libevent b. install boost c. install thrift d. generate mapkeeper binding step 3: After generating mapkeeper binding, go the "mapkeeper/thrift/gen-java/target", and run 

   cp mapkeeper-1.1-SNAPSHOT.jar mapkeeper-1.0.jar
   mvn install:install-file -Dfile=mapkeeper-1.0.jar -DgroupId=com.yahoo.mapkeeper -DartifactId=mapkeeper -Dversion=1.0 -Dpackaging=jar

The two commands run successfully step 4: Modify "YCSB/mapkeeper/pom.xml" as follows:

    <dependency>
           <groupId>com.yahoo.mapkeeper</groupId>
           <artifactId>mapkeeper</artifactId>
           <version>1.0</version>
    </dependency>
    <dependency>
        <groupId>org.apache.thrift</groupId>
        <artifactId>libthrift</artifactId>
        <version>0.8.0</version>
    </dependency>

step 5: Uncomment <module>mapkeeper</module> and <mapkeeper.version>1.0</mapkeeper.version> in "YCSB/pom.xml" step 6: reRun mvn clean package in YCSB directory

I got the outputs in step 6:

I seem to get the artifact issue in the latest version(0.12.0). After I run the "mvn clean package", I get the outputs: [INFO] Reactor Summary: [INFO] [INFO] YCSB Root .......................................... SUCCESS [ 2.010 s] [INFO] Core YCSB .......................................... SUCCESS [ 13.360 s] [INFO] Per Datastore Binding descriptor ................... SUCCESS [ 0.419 s] [INFO] YCSB Datastore Binding Parent ...................... SUCCESS [ 0.832 s] [INFO] Accumulo DB Binding ................................ SUCCESS [06:10 min] [INFO] Aerospike DB Binding ............................... SUCCESS [ 7.636 s] [INFO] ArangoDB Binding ................................... SUCCESS [ 2.068 s] [INFO] AsyncHBase Client Binding for Apache HBase ......... SUCCESS [01:01 min] [INFO] Azure table storage Binding ........................ SUCCESS [ 1.382 s] [INFO] Cassandra 2.1+ DB Binding .......................... SUCCESS [ 52.470 s] [INFO] Couchbase Binding .................................. SUCCESS [ 2.999 s] [INFO] Couchbase Java SDK 2.x Binding ..................... SUCCESS [ 2.365 s] [INFO] Azure DocumentDB Binding ........................... SUCCESS [ 0.926 s] [INFO] DynamoDB DB Binding ................................ SUCCESS [ 15.698 s] [INFO] Elasticsearch Binding .............................. SUCCESS [ 19.951 s] [INFO] Geode DB Binding ................................... SUCCESS [ 8.180 s] [INFO] Google Cloud Datastore Binding ..................... SUCCESS [ 2.674 s] [INFO] Google Cloud Bigtable Binding ...................... SUCCESS [ 8.763 s] [INFO] HBase 0.98.x DB Binding ............................ SUCCESS [ 7.534 s] [INFO] HBase 0.94.x DB Binding ............................ SUCCESS [ 8.953 s] [INFO] HBase 1.0 DB Binding ............................... SUCCESS [01:04 min] [INFO] Hypertable DB Binding .............................. SUCCESS [ 2.535 s] [INFO] Infinispan DB Binding .............................. SUCCESS [ 2.284 s] [INFO] JDBC DB Binding .................................... SUCCESS [ 4.903 s] [INFO] Kudu DB Binding .................................... SUCCESS [ 2.029 s] [INFO] memcached binding .................................. SUCCESS [ 1.775 s] [INFO] MongoDB Binding .................................... SUCCESS [ 4.154 s] [INFO] Oracle NoSQL Database Binding ...................... SUCCESS [ 1.138 s] [INFO] OrientDB Binding ................................... SUCCESS [ 8.265 s] [INFO] rados of Ceph FS binding ........................... SUCCESS [ 2.524 s] [INFO] Redis DB Binding ................................... SUCCESS [ 1.020 s] [INFO] Rest Client Binding ................................ SUCCESS [ 27.616 s] [INFO] Riak KV Binding .................................... SUCCESS [ 6.584 s] [INFO] S3 Storage Binding ................................. SUCCESS [ 1.833 s] [INFO] Solr Binding ....................................... SUCCESS [01:29 min] [INFO] Solr 6 Binding ..................................... SUCCESS [01:45 min] [INFO] Tarantool DB Binding ............................... SUCCESS [ 1.562 s] [INFO] YCSB Release Distribution Builder .................. FAILURE [ 4.023 s] [INFO] Mapkeeper DB Binding ............................... SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 15:23 min [INFO] Finished at: 2016-12-21T16:11:39+08:00 [INFO] Final Memory: 124M/237M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:2.5.5:single (default) on project ycsb: Failed to create assembly: Artifact: com.yahoo.ycsb:mapkeeper-binding:jar:0.13.0-SNAPSHOT (included by module) does not have an artifact with a file. Please ensure the package phase is run before the assembly is generated. -> [Help 1]

How can I solve this problem?

For clarity of the description, I attach the two pom.xml files: YCSB/pom.xml and
YCSB/mapkeeper/pom.xml. (The .xml file type is not supported to be uploaded, so, I change the file type from .xml to .txt here)

pom (YCSB).txt pom(mapkeeper).txt

I tried the not only the latest version 012.0 but also the old version(0.11.0 and 0.10.0) . They all failed because of the same reason.

risdenk commented 7 years ago

You are getting [INFO] Mapkeeper DB Binding ............................... SKIPPED that to me is saying its not trying to build the mapkeeper db binding.

It looks like you have to add the following to distribution/pom.xml:

<dependency>
  <groupId>com.yahoo.ycsb</groupId>
  <artifactId>mapkeeper-binding</artifactId>
  <version>${project.version}</version>
</dependency>

Can you try the following from the YCSB directory: mvn clean; mvn package -pl mapkeeper-binding -am

CaixinGong commented 7 years ago

I still failed because of other errors, after changing the Step 5 and Step 6 as follows. Step 5: Uncomment mapkeeper and 1.0</mapkeeper.version> in "YCSB/pom.xml"; add the following to distribution/pom.xml:

<dependency>
  <groupId>com.yahoo.ycsb</groupId>
  <artifactId>mapkeeper-binding</artifactId>
  <version>${project.version}</version>
</dependency>

Step 6: 6.1, run mvn cleansuccessfully, 6.2 different commands all failed. 6.2.1 run mvn clean; mvn package -pl mapkeeper-binding -am , I got the output: [ERROR] [ERROR] Could not find the selected project in the reactor: mapkeeper-binding @ [ERROR] Could not find the selected project in the reactor: mapkeeper-binding -> [Help 1]

6.2.2 You may mean to run mvn clean; mvn -pl com.yahoo.ycsb:mapkeeper-binding -am clean package, I got the output:

[ERROR] src/main/java/com/yahoo/ycsb/db/MapKeeperClient.java[0] (javadoc) JavadocPackage: Missing package-info.java file. [ERROR] src/main/java/com/yahoo/ycsb/db/MapKeeperClient.java[187:17] (extension) TreeWalker: Got an exception - expecting EOF, found '}' [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] YCSB Root .......................................... SUCCESS [ 2.437 s] [INFO] Core YCSB .......................................... SUCCESS [ 10.347 s] [INFO] Per Datastore Binding descriptor ................... SUCCESS [ 0.426 s] [INFO] YCSB Datastore Binding Parent ...................... SUCCESS [ 0.949 s] [INFO] Mapkeeper DB Binding ............................... FAILURE [ 0.459 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 15.306 s [INFO] Finished at: 2016-12-26T10:00:57+08:00 [INFO] Final Memory: 37M/589M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:2.16:check (validate) on project mapkeeper-binding: You have 2 Checkstyle violations. -> [Help 1]

6.2.3 After run mvn clean package, I got the output: [INFO] There are 2 errors reported by Checkstyle 6.2 with checkstyle.xml ruleset. [ERROR] src/main/java/com/yahoo/ycsb/db/MapKeeperClient.java[0] (javadoc) JavadocPackage: Missing package-info.java file. [ERROR] src/main/java/com/yahoo/ycsb/db/MapKeeperClient.java[187:17] (extension) TreeWalker: Got an exception - expecting EOF, found '}' [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] YCSB Root .......................................... SUCCESS [ 1.565 s] [INFO] Core YCSB .......................................... SUCCESS [ 7.751 s] [INFO] Per Datastore Binding descriptor ................... SUCCESS [ 0.310 s] [INFO] YCSB Datastore Binding Parent ...................... SUCCESS [ 0.608 s] [INFO] Accumulo DB Binding ................................ SUCCESS [05:23 min] [INFO] Aerospike DB Binding ............................... SUCCESS [ 5.570 s] [INFO] ArangoDB Binding ................................... SUCCESS [ 1.477 s] [INFO] AsyncHBase Client Binding for Apache HBase ......... SUCCESS [02:33 min] [INFO] Azure table storage Binding ........................ SUCCESS [ 0.826 s] [INFO] Cassandra 2.1+ DB Binding .......................... SUCCESS [01:36 min] [INFO] Couchbase Binding .................................. SUCCESS [ 10.334 s] [INFO] Couchbase Java SDK 2.x Binding ..................... SUCCESS [ 7.370 s] [INFO] Azure DocumentDB Binding ........................... SUCCESS [ 2.501 s] [INFO] DynamoDB DB Binding ................................ SUCCESS [ 36.151 s] [INFO] Elasticsearch Binding .............................. SUCCESS [01:09 min] [INFO] Geode DB Binding ................................... SUCCESS [ 15.375 s] [INFO] Google Cloud Datastore Binding ..................... SUCCESS [ 17.363 s] [INFO] Google Cloud Bigtable Binding ...................... SUCCESS [ 28.209 s] [INFO] HBase 0.98.x DB Binding ............................ SUCCESS [ 17.536 s] [INFO] HBase 0.94.x DB Binding ............................ SUCCESS [ 14.672 s] [INFO] HBase 1.0 DB Binding ............................... SUCCESS [02:07 min] [INFO] Hypertable DB Binding .............................. SUCCESS [ 2.727 s] [INFO] Infinispan DB Binding .............................. SUCCESS [ 4.463 s] [INFO] JDBC DB Binding .................................... SUCCESS [ 15.285 s] [INFO] Kudu DB Binding .................................... SUCCESS [ 3.029 s] [INFO] memcached binding .................................. SUCCESS [ 2.389 s] [INFO] Mapkeeper DB Binding ............................... FAILURE [ 1.827 s] [INFO] MongoDB Binding .................................... SKIPPED [INFO] Oracle NoSQL Database Binding ...................... SKIPPED [INFO] OrientDB Binding ................................... SKIPPED [INFO] rados of Ceph FS binding ........................... SKIPPED [INFO] Redis DB Binding ................................... SKIPPED [INFO] Rest Client Binding ................................ SKIPPED [INFO] Riak KV Binding .................................... SKIPPED [INFO] S3 Storage Binding ................................. SKIPPED [INFO] Solr Binding ....................................... SKIPPED [INFO] Solr 6 Binding ..................................... SKIPPED [INFO] Tarantool DB Binding ............................... SKIPPED [INFO] YCSB Release Distribution Builder .................. SKIPPED [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 16:07 min [INFO] Finished at: 2016-12-26T09:50:18+08:00 [INFO] Final Memory: 98M/233M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:2.16:check (validate) on project mapkeeper-binding: You have 2 Checkstyle violations. -> [Help 1]

I am very confused for the things below: It seems the YCSB/mapkeeper/src/main/java/com/yahoo/ycsb/db directory lost package-info.java file, so I copy the YCSB/aerospike/src/main/java/com/yahoo/ycsb/db/package-info.java to the directory. The first error was solved, but, the second error still exists.

Line 186 of YCSB/mapkeeper/src/main/java/com/yahoo/ycsb/db/package-info.java file has an extra bracket， which make the second error occurs. What's more, more errors occur when I delete the extra bracket and runmvn -pl com.yahoo.ycsb:mapkeeper-binding -am clean package(mvn clean packagecommand get the same errors): [ERROR] src/main/java/com/yahoo/ycsb/db/MapKeeperClient.java[44] (javadoc) JavadocType: Missing a Javadoc comment. [ERROR] src/main/java/com/yahoo/ycsb/db/MapKeeperClient.java[45] (indentation) Indentation: 'member def modifier' have incorrect indentation level 4, expected level should be 2. [ERROR] src/main/java/com/yahoo/ycsb/db/MapKeeperClient.java[46] (indentation) Indentation: 'member def modifier' have incorrect indentation level 4, expected level should be 2. [ERROR] src/main/java/com/yahoo/ycsb/db/MapKeeperClient.java[47] (indentation) Indentation:

... ...(Omitted)

[ERROR] src/main/java/com/yahoo/ycsb/db/MapKeeperClient.java[217] (indentation) Indentation: 'catch rcurly' have incorrect indentation level 8, expected level should be 4. [ERROR] src/main/java/com/yahoo/ycsb/db/MapKeeperClient.java[218] (indentation) Indentation: 'method def rcurly' have incorrect indentation level 4, expected level should be 2. [INFO] ------------------------------------------------------------------------ [INFO] Reactor Summary: [INFO] [INFO] YCSB Root .......................................... SUCCESS [ 2.461 s] [INFO] Core YCSB .......................................... SUCCESS [ 10.465 s] [INFO] Per Datastore Binding descriptor ................... SUCCESS [ 0.418 s] [INFO] YCSB Datastore Binding Parent ...................... SUCCESS [ 0.946 s] [INFO] Mapkeeper DB Binding ............................... FAILURE [ 0.496 s] [INFO] ------------------------------------------------------------------------ [INFO] BUILD FAILURE [INFO] ------------------------------------------------------------------------ [INFO] Total time: 15.474 s [INFO] Finished at: 2016-12-26T13:14:18+08:00 [INFO] Final Memory: 38M/590M [INFO] ------------------------------------------------------------------------ [ERROR] Failed to execute goal org.apache.maven.plugins:maven-checkstyle-plugin:2.16:check (validate) on project mapkeeper-binding: You have 179 Checkstyle violations. -> [Help 1]

Looking forward to your reply!

risdenk commented 7 years ago

The mapkeeper-binding has some check style validation issues. Since the mapkeeper-binding was added, YCSB added some check style checks for conformance across the bindings. You can fix the validation issues. The validation issues are in the Maven output you posted. You are making progress since the binding now is being built.

CaixinGong commented 7 years ago

I successfully intall mapkeeper when doing the six steps on YCSB-0.4.0. Thank you very much. I am very poor in maven, I may try to fix the validation issues latter.