mapreduce-bitcoinblock test fails

[rounsavall]

mapreduce-bitcoinblock test fails with:

./gradlew clean build --debug ...

Task :examples:mapreduce-bitcoinblock:testIntegration

org.zuinnote.hadoop.bitcoin.example.MapReduceBitcoinBlockIntegrationTest > mapReduceGenesisBlock() FAILED org.opentest4j.AssertionFailedError at MapReduceBitcoinBlockIntegrationTest.java:189

2 tests completed, 1 failed

Task :examples:mapreduce-bitcoinblock:testIntegration FAILED

FAILURE: Build failed with an exception.

• What went wrong: Execution failed for task ':examples:mapreduce-bitcoinblock:testIntegration'.

  There were failing tests. See the report at: file:///Users/rounsavall/hadoopcryptoledger/examples/mapreduce-bitcoinblock/build/reports/tests/testIntegration/index.html

• Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

For more detailed output, check application tracking page:http://rounsavall-mpb15.local:61000/cluster/app/application_1538687349113_0001Then, click on links to logs of each attempt. Diagnostics: Exception from container-launch. Container id: container_1538687349113_0001_02_000001 Exit code: 127 Stack trace: ExitCodeException exitCode=127: at org.apache.hadoop.util.Shell.runCommand(Shell.java:545) at org.apache.hadoop.util.Shell.run(Shell.java:456) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

Container exited with a non-zero exit code 127 Failing this attempt. Failing the application. 2018-10-04 14:09:19,076 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(721)) - application_1538687349113_0001 State change from FINAL_SAVING to FAILED 2018-10-04 14:09:19,077 WARN [AsyncDispatcher event handler] resourcemanager.RMAuditLogger (RMAuditLogger.java:logFailure(263)) - USER=rounsavall OPERATION=Application Finished - Failed TARGET=RMAppManager RESULT=FAILURE DESCRIPTION=App failed with state: FAILED PERMISSIONS=Application application_1538687349113_0001 failed 2 times due to AM Container for appattempt_1538687349113_0001_000002 exited with exitCode: 127 For more detailed output, check application tracking page:http://rounsavall-mpb15.local:61000/cluster/app/application_1538687349113_0001Then, click on links to logs of each attempt. Diagnostics: Exception from container-launch. Container id: container_1538687349113_0001_02_000001 Exit code: 127 Stack trace: ExitCodeException exitCode=127: at org.apache.hadoop.util.Shell.runCommand(Shell.java:545) at org.apache.hadoop.util.Shell.run(Shell.java:456) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

Container exited with a non-zero exit code 127 Failing this attempt. Failing the application. APPID=application_1538687349113_0001 2018-10-04 14:09:19,078 INFO [AsyncDispatcher event handler] resourcemanager.RMAppManager$ApplicationSummary (RMAppManager.java:logAppSummary(187)) - appId=application_1538687349113_0001,name=example-hadoop-bitcoin-transactioncounter-job,user=rounsavall,queue=default,state=FAILED,trackingUrl=http://rounsavall-mpb15.local:61000/cluster/app/application_1538687349113_0001,appMasterHost=N/A,startTime=1538687354509,finishTime=1538687359074,finalStatus=FAILED,memorySeconds=4604,vcoreSeconds=2,preemptedAMContainers=0,preemptedNonAMContainers=0,preemptedResources=<memory:0\, vCores:0>,applicationType=MAPREDUCE 2018-10-04 14:09:19,834 INFO [Test worker] mapreduce.Job (Job.java:monitorAndPrintJob(1360)) - Job job_1538687349113_0001 running in uber mode : false 2018-10-04 14:09:19,834 INFO [Test worker] mapreduce.Job (Job.java:monitorAndPrintJob(1367)) - map 0% reduce 0% 2018-10-04 14:09:19,846 INFO [Test worker] mapreduce.Job (Job.java:monitorAndPrintJob(1380)) - Job job_1538687349113_0001 failed with state FAILED due to: Application application_1538687349113_0001 failed 2 times due to AM Container for appattempt_1538687349113_0001_000002 exited with exitCode: 127 For more detailed output, check application tracking page:http://rounsavall-mpb15.local:61000/cluster/app/application_1538687349113_0001Then, click on links to logs of each attempt. Diagnostics: Exception from container-launch. Container id: container_1538687349113_0001_02_000001 Exit code: 127 Stack trace: ExitCodeException exitCode=127: at org.apache.hadoop.util.Shell.runCommand(Shell.java:545) at org.apache.hadoop.util.Shell.run(Shell.java:456) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

Container exited with a non-zero exit code 127 Failing this attempt. Failing the application.

[jornfranke]

Which operating system are you using?

On Thu, Oct 4, 2018 at 11:44 PM rounsavall notifications@github.com wrote:

mapreduce-bitcoinblock test fails with:

Task :examples:mapreduce-bitcoinblock:testIntegration

org.zuinnote.hadoop.bitcoin.example.MapReduceBitcoinBlockIntegrationTest > mapReduceGenesisBlock() FAILED org.opentest4j.AssertionFailedError at MapReduceBitcoinBlockIntegrationTest.java:189

2 tests completed, 1 failed

Task :examples:mapreduce-bitcoinblock:testIntegration FAILED

FAILURE: Build failed with an exception.

• What went wrong: Execution failed for task ':examples:mapreduce-bitcoinblock:testIntegration'.

There were failing tests. See the report at: file:///Users/rounsavall/hadoopcryptoledger/examples/mapreduce-bitcoinblock/build/reports/tests/testIntegration/index.html

• Try: Run with --stacktrace option to get the stack trace. Run with --info or --debug option to get more log output. Run with --scan to get full insights.

For more detailed output, check application tracking page: http://axbarreto.local:61000/cluster/app/application_1538687349113_0001Then, click on links to logs of each attempt. Diagnostics: Exception from container-launch. Container id: container_1538687349113_0001_02_000001 Exit code: 127 Stack trace: ExitCodeException exitCode=127: at org.apache.hadoop.util.Shell.runCommand(Shell.java:545) at org.apache.hadoop.util.Shell.run(Shell.java:456) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

Container exited with a non-zero exit code 127 Failing this attempt. Failing the application. 2018-10-04 14:09:19,076 INFO [AsyncDispatcher event handler] rmapp.RMAppImpl (RMAppImpl.java:handle(721)) - application_1538687349113_0001 State change from FINAL_SAVING to FAILED 2018-10-04 14:09:19,077 WARN [AsyncDispatcher event handler] resourcemanager.RMAuditLogger (RMAuditLogger.java:logFailure(263)) - USER=axbarreto OPERATION=Application Finished - Failed TARGET=RMAppManager RESULT=FAILURE DESCRIPTION=App failed with state: FAILED PERMISSIONS=Application application_1538687349113_0001 failed 2 times due to AM Container for appattempt_1538687349113_0001_000002 exited with exitCode: 127 For more detailed output, check application tracking page: http://axbarreto.local:61000/cluster/app/application_1538687349113_0001Then, click on links to logs of each attempt. Diagnostics: Exception from container-launch. Container id: container_1538687349113_0001_02_000001 Exit code: 127 Stack trace: ExitCodeException exitCode=127: at org.apache.hadoop.util.Shell.runCommand(Shell.java:545) at org.apache.hadoop.util.Shell.run(Shell.java:456) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

Container exited with a non-zero exit code 127 Failing this attempt. Failing the application. APPID=application_1538687349113_0001 2018-10-04 14:09:19,078 INFO [AsyncDispatcher event handler] resourcemanager.RMAppManager$ApplicationSummary (RMAppManager.java:logAppSummary(187)) - appId=application_1538687349113_0001,name=example-hadoop-bitcoin-transactioncounter-job,user=rounsacall,queue=default,state=FAILED,trackingUrl= http://axbarreto.local:61000/cluster/app/application_1538687349113_0001,appMasterHost=N/A,startTime=1538687354509,finishTime=1538687359074,finalStatus=FAILED,memorySeconds=4604,vcoreSeconds=2,preemptedAMContainers=0,preemptedNonAMContainers=0,preemptedResources=<memory:0, vCores:0>,applicationType=MAPREDUCE 2018-10-04 14:09:19,834 INFO [Test worker] mapreduce.Job (Job.java:monitorAndPrintJob(1360)) - Job job_1538687349113_0001 running in uber mode : false 2018-10-04 14:09:19,834 INFO [Test worker] mapreduce.Job (Job.java:monitorAndPrintJob(1367)) - map 0% reduce 0% 2018-10-04 14:09:19,846 INFO [Test worker] mapreduce.Job (Job.java:monitorAndPrintJob(1380)) - Job job_1538687349113_0001 failed with state FAILED due to: Application application_1538687349113_0001 failed 2 times due to AM Container for appattempt_1538687349113_0001_000002 exited with exitCode: 127 For more detailed output, check application tracking page: http://rounsavall-mpb15.local:61000/cluster/app/application_1538687349113_0001Then, click on links to logs of each attempt. Diagnostics: Exception from container-launch. Container id: container_1538687349113_0001_02_000001 Exit code: 127 Stack trace: ExitCodeException exitCode=127: at org.apache.hadoop.util.Shell.runCommand(Shell.java:545) at org.apache.hadoop.util.Shell.run(Shell.java:456) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:722) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:211) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:302) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:82) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)

Container exited with a non-zero exit code 127 Failing this attempt. Failing the application.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ZuInnoTe/hadoopcryptoledger/issues/62, or mute the thread https://github.com/notifications/unsubscribe-auth/ABlR5DPLjiwNCSL957mEogeLsglAYadHks5uhoE3gaJpZM4XI94r .

[jornfranke]

The reason I asked is that the Hadoop/Yarn Minicluster integration does ONLY support Linux. From your excerpt I conclude that you are using MacOs. This is not supported by the integration tests on Hadoop/Yarn minicluster - if you want to change this you have to open an issue in the Hadoop project. Nevertheless, that does not forbid you to develop on MacOs and run the application on a cluster. It runs perfectly fine there. If you require integration tests (which you should as part of a good CI pipeline) you need to resort to a Linux based environment.

[jornfranke]

did this help?

[jornfranke]

Please open a new issue in case problems reappear.