zyhxq commented 8 years ago

一、删除中文乱码文件夹 find . -inum 55711317 -print -exec rm {} -rf \;

zyhxq commented 8 years ago

nl [选项] [文件] 显示文件内容加行号常用参数：

参数描述 -b 指定行号指定的方式，主要有两种： -b a 表示不论是否为空行，也同样列出行号(类似 cat -n) -b t 如果有空行，空的那一行不要列出行号(默认值) -n 列出行号表示的方法，主要有三种： -n ln 行号在屏幕的最左方显示 -n rn 行号在自己栏位的最右方显示，且不加 0 -n rz 行号在自己栏位的最右方显示，且加 0 -w 行号栏位的占用的位数

zyhxq commented 8 years ago

添加用户 useradd hadoop 将某个用户添加到用户组 usermod -G groupname username 如将hadoop添加到sudo用户组 usermod -G sudo hadoop

zyhxq commented 8 years ago

$ sudo apt-get update $ sudo apt-get install openssh-server rsync

zyhxq commented 8 years ago

$ sudo apt-get install openjdk-7-jdk $ java -version

zyhxq commented 8 years ago

切换用户换到hadoop用户，需要输入添加hadoop用户时配置的密码。后续步骤都将在hadoop用户的环境中执行。 su -l hadoop

zyhxq commented 8 years ago

配置ssh环境免密码登录。 ssh-keygen -t rsa -P "" 在/home/hadoop/.ssh目录下生成了id_rsa（私钥）和id_rsa.pub（公钥）两个文件,将公钥追加到authorized_keys中，该文件保存所有允许以当前用户身份登录到ssh客户端用户的公钥内容。 cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys 验证登录本机是否还需要密码，配置正确的话是可以不需密码登录的。 ssh localhost

zyhxq commented 8 years ago

下载并安装hadoop 2.6.0 wget http://labfile.oss.aliyuncs.com/hadoop-2.6.0.tar.gz

zyhxq commented 8 years ago

HADOOP START

export JAVA_HOME=/usr/lib/jvm/java-7-openjdk-amd64 export HADOOP_INSTALL=/usr/local/hadoop export PATH=$PATH:$HADOOP_INSTALL/bin export PATH=$PATH:$HADOOP_INSTALL/sbin export HADOOP_MAPRED_HOME=$HADOOP_INSTALL export HADOOP_COMMON_HOME=$HADOOP_INSTALL export HADOOP_HDFS_HOME=$HADOOP_INSTALL export YARN_HOME=$HADOOP_INSTALL export HADOOP_COMMON_LIB_NATIVE_DIR=$HADOOP_INSTALL/lib/native export HADOOP_OPTS="-Djava.library.path=$HADOOP_INSTALL/lib"

HADOOP END

zyhxq commented 8 years ago

词频统计 $ bin/hadoop jar share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-2.6.0-sources.jar org.apache.hadoop.examples.WordCount input output

zyhxq commented 8 years ago

查看操作系统版本 lsb_release -a LSB Version: :core-4.0-amd64:core-4.0-ia32:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-ia32:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-ia32:printing-4.0-noarch Distributor ID: AlibabaGroupEnterpriseServer Description: Alibaba Group Enterprise Linux Server release 5.7 (CatFeces) Release: 5.7 Codename: CatFeces

zyhxq commented 8 years ago

uname -a Linux e100069199183.zmf 2.6.32-220.23.2.ali927.el5.x86_64 #1 SMP Mon Jan 28 14:57:06 CST 2013 x86_64 x86_64 x86_64 GNU/Linux

zyhxq / techbook

linux命令实用大全 #1

HADOOP START

HADOOP END