ZuInnoTe / hadoopcryptoledger

Hadoop Crypto Ledger - Analyzing CryptoLedgers, such as Bitcoin Blockchain, on Big Data platforms, such as Hadoop/Spark/Flink/Hive
Apache License 2.0
141 stars 51 forks source link

gradle problem:testIntegeration maoReduceGenesisBlock() fail #55

Closed foonsun closed 6 years ago

foonsun commented 6 years ago

@jornfranke Hi,thanks for your great project.I occur this issue when run gradlew build.

╷
├─ JMockit integration ✔tcoinblock:testIntegration
└─ JUnit Jupiter ✔
   └─ MapReduceBitcoinBlockIntegrationTest ✔
      ├─ mapReduceGenesisBlock() ✘ Successfully executed mapreduce application ==> expected: <0> but was: <1>
      └─ checkTestDataGenesisBlockAvailable() ✔

Failures (1):
  JUnit Jupiter:MapReduceBitcoinBlockIntegrationTest:mapReduceGenesisBlock()
    MethodSource [className = 'org.zuinnote.hadoop.bitcoin.example.MapReduceBitcoinBlockIntegrationTest', methodName = 'mapReduceGenesisBlock', methodParameterTypes = '']
    => org.opentest4j.AssertionFailedError: Successfully executed mapreduce application ==> expected: <0> but was: <1>
       org.zuinnote.hadoop.bitcoin.example.MapReduceBitcoinBlockIntegrationTest.mapReduceGenesisBlock(MapReduceBitcoinBlockIntegrationTest.java:187)

I have tried over for one day.but still i can't work over it.I need your help. thanks very much.

My jdk is 1.8.0_171 hadoop version

Hadoop 2.6.0-cdh5.14.2
Subversion http://github.com/cloudera/hadoop -r 5724a4ad7a27f7af31aa725694d3df09a68bb213
Compiled by jenkins on 2018-03-27T20:40Z
Compiled with protoc 2.5.0
From source with checksum 302899e86485742c090f626a828b28
This command was run using /usr/lib/hadoop/hadoop-common-2.6.0-cdh5.14.2.jar
spark-shell --version
Welcome to
      ____              __
     / __/__  ___ _____/ /__
    _\ \/ _ \/ _ `/ __/  '_/
   /___/ .__/\_,_/_/ /_/\_\   version 1.6.0
      /_/
jornfranke commented 6 years ago

Are you using MacOS as a build platform?

On 7. Jun 2018, at 05:05, foonsun notifications@github.com wrote:

Hi,thanks for your great project.I occur this issue when run gradlew build.

╷ ├─ JMockit integration ✔tcoinblock:testIntegration └─ JUnit Jupiter ✔ └─ MapReduceBitcoinBlockIntegrationTest ✔ ├─ mapReduceGenesisBlock() ✘ Successfully executed mapreduce application ==> expected: <0> but was: <1> └─ checkTestDataGenesisBlockAvailable() ✔

Failures (1): JUnit Jupiter:MapReduceBitcoinBlockIntegrationTest:mapReduceGenesisBlock() MethodSource [className = 'org.zuinnote.hadoop.bitcoin.example.MapReduceBitcoinBlockIntegrationTest', methodName = 'mapReduceGenesisBlock', methodParameterTypes = ''] => org.opentest4j.AssertionFailedError: Successfully executed mapreduce application ==> expected: <0> but was: <1> org.zuinnote.hadoop.bitcoin.example.MapReduceBitcoinBlockIntegrationTest.mapReduceGenesisBlock(MapReduceBitcoinBlockIntegrationTest.java:187) — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

foonsun commented 6 years ago

@jornfranke no,i am using Cloudera Quickstart docker images (HDP®) 2.6.4 on aws.

jornfranke commented 6 years ago

Well until now this integration test failed only on MacOs. On Ubuntu it went through and as well on Travis (our CI infrastructure in the cloud) - as you can see from the CI platform. Since most of the clusters run on Linux, we did not deeper investigate. I think it is just an issue in the Hadoop Cluster libraries for unit testing (MiniyarnCluster). You can deactivate this integrationTest for now and I will check if I find the root cause. Deactivate it by puttting @Disable in front of the method mapReduceGenesisBlockTest (https://github.com/ZuInnoTe/hadoopcryptoledger/blob/master/examples/mapreduce-bitcoinblock/src/integration-test/java/org/zuinnote/hadoop/bitcoin/example/MapReduceBitcoinBlockIntegrationTest.java). It will build fine and run fine (we tested it on HDP) and you find the jar in build/libs. All non-Mapreduce applications do not show this issue from our experience.

On 7. Jun 2018, at 07:08, foonsun notifications@github.com wrote:

@jornfranke no,i am using Cloudera Quickstart docker images (HDP®) 2.6.4 on aws.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

foonsun commented 6 years ago

@jornfranke thanks for your great help.I will try to disable this test.

jornfranke commented 6 years ago

Thx for the feedback. Let me know how it turns out. Maybe I will rewrite this test anyway because the problematic assertion does not really test the Hadoopcryptoledger library - more interesting is the test that follows after. In any case it is just a test of the example, the unit tests of the library itself work without problems.

On 7. Jun 2018, at 07:50, foonsun notifications@github.com wrote:

@jornfranke thanks for your great help.I will try to disable this test.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

foonsun commented 6 years ago

yes.it runs well.thanks very much for your help.