search

RevolutionAnalytics / RHadoop

RHadoop
https://github.com/RevolutionAnalytics/RHadoop/wiki
763 stars 278 forks source link

Error: ignoring SIGPIPE signal #209

Open ddskyfuyu opened 10 years ago

ddskyfuyu commented 10 years ago

Hi, I use the "Finding Frequent Itemsets" example in my hadoop cluster, but I got a problem. I don't know how to solve this problem. Please help me, thanks.

stderr Log:

Loading required package: rhdfs library(package, lib.loc = lib.loc, character.only = TRUE, logical.return = TRUE, 里有警告: there is no package called ‘rhdfs’ FUN(c("rmr2", "caTools", "plyr", "stringr", "reshape2", "functional", 里有警告: can't load rhdfs Loading required package: rJava ERROR : ignoring SIGPIPE signal No traceback available Error during wrapup: Execution halted java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1 at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:362) at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:576) at org.apache.hadoop.streaming.PipeMapper.map(PipeMapper.java:91) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) at org.apache.hadoop.mapred.Child.main(Child.java:249) java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1 at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:362) at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:576) at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:135) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57) at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) at org.apache.hadoop.mapred.Child.main(Child.java:249)

piccolbo commented 10 years ago

Please file in the issue tracker for the package you are reporting about. Please follow bug reporting guidelines.

Antonio

On Wed, May 21, 2014 at 10:43 PM, ddskyfuyu notifications@github.comwrote:

Hi, I use the "Finding Frequent Itemsets" example in my hadoop cluster, but I got a problem. I don't know how to solve this problem. Please help me, thanks.

stderr Log:

Loading required package: rhdfs library(package, lib.loc = lib.loc, character.only = TRUE, logical.return = TRUE, 里有警告: there is no package called ‘rhdfs’ FUN(c("rmr2", "caTools", "plyr", "stringr", "reshape2", "functional", 里有警告: can't load rhdfs Loading required package: rJava ERROR : ignoring SIGPIPE signal No traceback available Error during wrapup: Execution halted java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1 at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:362) at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:576) at org.apache.hadoop.streaming.PipeMapper.map(PipeMapper.java:91) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) at org.apache.hadoop.mapred.Child.main(Child.java:249) java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1 at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:362) at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:576) at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:135) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57) at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) at org.apache.hadoop.mapred.Child.main(Child.java:249)

— Reply to this email directly or view it on GitHubhttps://github.com/RevolutionAnalytics/RHadoop/issues/209 .

ddskyfuyu commented 10 years ago

Hi, I uninstall the rhdfs, and install rhdfs again. The "ignoring SIGPIPE signal" problem don't appear. When I use a very small dataset(5000), the "Finding Frequent Itemsets" example still failed.

The error message:

Error : .onLoad failed in loadNamespace() for 'rhdfs', details: call: fun(libname, pkgname) error: Environment variable HADOOP_CMD must be set before loading package rhdfs

java.lang.RuntimeException: java.lang.OutOfMemoryError: Java heap space at org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:376) at org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:576) at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:135) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57) at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) at org.apache.hadoop.mapred.Child.main(Child.java:249) Caused by: java.lang.OutOfMemoryError: Java heap space at org.apache.hadoop.typedbytes.TypedBytesInput.readRawBytes(TypedBytesInput.java:212) at org.apache.hadoop.typedbytes.TypedBytesInput.readRaw(TypedBytesInput.java:152) at org.apache.hadoop.typedbytes.TypedBytesInput.readRawMap(TypedBytesInput.java:489) at org.apache.hadoop.typedbytes.TypedBytesInput.readRaw(TypedBytesInput.java:148) at org.apache.hadoop.streaming.io.TypedBytesOutputReader.readKeyValue(TypedBytesOutputReader.java:51) at org.apache.hadoop.streaming.PipeMapRed$MROutputThread.run(PipeMapRed.java:418)

2014-05-22 13:55 GMT+08:00 Antonio Piccolboni notifications@github.com:

Please file in the issue tracker for the package you are reporting about. Please follow bug reporting guidelines.

Antonio

On Wed, May 21, 2014 at 10:43 PM, ddskyfuyu notifications@github.comwrote:

Hi, I use the "Finding Frequent Itemsets" example in my hadoop cluster, but I got a problem. I don't know how to solve this problem. Please help me, thanks.

stderr Log:

Loading required package: rhdfs library(package, lib.loc = lib.loc, character.only = TRUE, logical.return = TRUE, 里有警告: there is no package called ‘rhdfs’ FUN(c("rmr2", "caTools", "plyr", "stringr", "reshape2", "functional", 里有警告: can't load rhdfs Loading required package: rJava ERROR : ignoring SIGPIPE signal No traceback available Error during wrapup: Execution halted java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1 at

org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:362)

at

org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:576) at org.apache.hadoop.streaming.PipeMapper.map(PipeMapper.java:91) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50) at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at

org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)

at org.apache.hadoop.mapred.Child.main(Child.java:249) java.lang.RuntimeException: PipeMapRed.waitOutputThreads(): subprocess failed with code 1 at

org.apache.hadoop.streaming.PipeMapRed.waitOutputThreads(PipeMapRed.java:362)

at

org.apache.hadoop.streaming.PipeMapRed.mapRedFinished(PipeMapRed.java:576) at org.apache.hadoop.streaming.PipeMapper.close(PipeMapper.java:135) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:57) at org.apache.hadoop.streaming.PipeMapRunner.run(PipeMapRunner.java:36) at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:436) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:372) at org.apache.hadoop.mapred.Child$4.run(Child.java:255) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at

org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)

at org.apache.hadoop.mapred.Child.main(Child.java:249)

— Reply to this email directly or view it on GitHub< https://github.com/RevolutionAnalytics/RHadoop/issues/209> .

— Reply to this email directly or view it on GitHubhttps://github.com/RevolutionAnalytics/RHadoop/issues/209#issuecomment-43851072 .