CocoaWang
commented
8 years ago Mabe you can try this solution: You need to setup ssh and set ssh login to No password mode.For we use hdfs in localhost.
Open rax845163 opened 8 years ago
CocoaWang
commented
8 years ago Mabe you can try this solution: You need to setup ssh and set ssh login to No password mode.For we use hdfs in localhost.
I'm a hadoop beginer. My college professor gave me a user which can login into a cloudera manager. I'm just getting started on HiBench and configuring the environment by this web page https://github.com/intel-hadoop/HiBench/wiki/Getting-Started
And now I encounter two question when running the prepared.sh under the "HiBench/workloads/wordcount/prepare/" diectory, hope anyone can give me a help.
1.The error message is
Exception in thread "main" java.lang.RuntimeException : core-site.xml not found. Before this error , there are two descriptions on the console .
start HadoopPrepareWordcount bench hdfs rm -r: /opt/cloudera/parcels/CDH//bin/hadoop --config /opt/cloudera/parcels/CDH/etc/hadoop fs -rm -r -skipTrash hdfs://10.1.18.2/HiBench/Wordcount/Input
where should I set the value to let the core-site.xml been found?
2.The error message is org.apache.hadoop.security.AccessControlException: Permission denied: user=just1, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.AccessControlException): Permission denied: user=just1, access=WRITE, inode="/":hdfs:supergroup:drwxr-xr-x
Here are some descriptions before this error:
Submit MapReduce Job: /opt/cloudera/parcels/CDH//bin/hadoop --config /opt/cloudera/parcels/CDH/etc/hadoop jar /opt/cloudera/parcels/CDH/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar randomtextwriter -D mapreduce.randomtextwriter.totalbytes=3200000000 -D mapreduce.job.maps=12 -D mapreduce.job.reduces=6 -D mapreduce.output.fileoutputformat.compress=false hdfs://10.1.18.2/HiBench/Wordcount/Input /user/hdfs/just/hibench_output The authenticity of host 'localhost (127.0.0.1)' can't be established. ECDSA key fingerprint is 1b:f3:bc:b5:67:54:5e:98:17:22:bf:d8:d5:0d:ff:2c. Are you sure you want to continue connecting (yes/no)? 2016-04-15 17:11:19,055 INFO [main] Configuration.deprecation (Configuration.java:warnOnceIfDeprecated(1087)) - session.id is deprecated. Instead, use dfs.metrics.session-id 2016-04-15 17:11:20,517 WARN [Thread-5] mapred.LocalJobRunner (LocalJobRunner.java:run(560)) - job_local1494445656_0001
In the HDFS , user just1 only have the permission to write file under the "/user/hdfs/just/" directory. I have appened a string "/user/hdfs/just/hibench_output" back to the content "run-hadoop-job" which in the prepare.sh , but it seem like it doesn't work. I wonder how to change the outputpath to where just1 can output file.
Hope someone can give me some suggestion. Thanks.