Linux: scp | rsync | nc

[lihongjie0209]

SCP

scp是secure copy的简写，用于在Linux下进行远程拷贝文件的命令，和它类似的命令有cp，不过cp只是在本机进行拷贝不能跨服务器，而且scp传输是加密的。当你服务器硬盘变为只读 read only system时，用scp可以帮你把文件移出来。

注解

类似的工具有rsync；scp消耗资源少，不会提高多少系统负荷，在这一点上，rsync就远远不及它了。rsync比scp会快一点，但当小文件多的情况下，rsync会导致硬盘I/O非常高，而scp基本不影响系统正常使用。

18.1. 命令格式：

scp [参数] [原路径] [目标路径]

18.2. 命令参数：

• -1 强制scp命令使用协议ssh1
• -2 强制scp命令使用协议ssh2
• -4 强制scp命令只使用IPv4寻址
• -6 强制scp命令只使用IPv6寻址
• -B 使用批处理模式（传输过程中不询问传输口令或短语）
• -C 允许压缩。（将-C标志传递给ssh，从而打开压缩功能）
• -p 留原文件的修改时间，访问时间和访问权限。
• -q 不显示传输进度条。
• -r 递归复制整个目录。
• -v 详细方式显示输出。scp和ssh(1)会显示出整个过程的调试信息。这些信息用于调试连接，验证和配置问题。
• -c cipher 以cipher将数据传输进行加密，这个选项将直接传递给ssh。
• -F ssh_config 指定一个替代的ssh配置文件，此参数直接传递给ssh。
• -i identity_file 从指定文件中读取传输时使用的密钥文件，此参数直接传递给ssh。
• -l limit 限定用户所能使用的带宽，以Kbit/s为单位。
• -o ssh_option 如果习惯于使用ssh_config(5)中的参数传递方式，
• -P port 注意是大写的P, port是指定数据传输用到的端口号
• -S program 指定加密传输时所使用的程序。此程序必须能够理解ssh(1)的选项。

18.3. 使用说明

从本地服务器复制到远程服务器

复制文件:

$scp local_file remote_username@remote_ip:remote_folder
$scp local_file remote_username@remote_ip:remote_file
$scp local_file remote_ip:remote_folder
$scp local_file remote_ip:remote_file

指定了用户名，命令执行后需要输入用户密码；如果不指定用户名，命令执行后需要输入用户名和密码；

复制目录:

$scp -r local_folder remote_username@remote_ip:remote_folder
$scp -r local_folder remote_ip:remote_folder

第1个指定了用户名，命令执行后需要输入用户密码； 第2个没有指定用户名，命令执行后需要输入用户名和密码；

注解

从远程复制到本地的scp命令与上面的命令一样，只要将从本地复制到远程的命令后面2个参数互换顺序就行了。

18.4. 使用示例

实例1：从远处复制文件到本地目录

$scp root@10.6.159.147:/opt/soft/demo.tar /opt/soft/

说明： 从10.6.159.147机器上的/opt/soft/的目录中下载demo.tar 文件到本地/opt/soft/目录中

实例2：从远处复制到本地

$scp -r root@10.6.159.147:/opt/soft/test /opt/soft/

说明： 从10.6.159.147机器上的/opt/soft/中下载test目录到本地的/opt/soft/目录来。

实例3：上传本地文件到远程机器指定目录

$scp /opt/soft/demo.tar root@10.6.159.147:/opt/soft/scptest

说明： 复制本地opt/soft/目录下的文件demo.tar 到远程机器10.6.159.147的opt/soft/scptest目录

实例4：上传本地目录到远程机器指定目录

$scp -r /opt/soft/test root@10.6.159.147:/opt/soft/scptest

说明： 上传本地目录 /opt/soft/test到远程机器10.6.159.147上/opt/soft/scptest的目录中

[lihongjie0209]

rsync

rsync is a fast and versatile command-line utility for synchronizing files and directories between two locations over a remote shell, or from/to a remote Rsync daemon. It provides fast incremental file transfer by transferring only the differences between the source and the destination.

Rsync can be used for mirroring data, incremental backups, copying files between systems, and as a replacement for scp , sftp , and cp commands.

This article explains how to use rsync through practical examples and detailed explanations of the most common rsync options.

Installing Rsync

The rsync utility is pre-installed on most Linux distributions and macOS. If you don’t have rsync installed on your system, you can easily install it using your distribution’s package manager.

Install Rsync on Ubuntu and Debian

sudo apt install rsync

Install Rsync on CentOS and Fedora

sudo yum install rsync

Rsync Command Syntax

Before going into how to use the rsync command, let’s start by reviewing the basic syntax.

The rsync utility expressions take the following form:

Local to Local:  rsync [OPTION]... [SRC]... DEST
Local to Remote: rsync [OPTION]... [SRC]... [USER@]HOST:DEST
Remote to Local: rsync [OPTION]... [USER@]HOST:SRC... [DEST]

• OPTION - The rsync options .
• SRC - Source directory.
• DEST - Destination directory.
• USER - Remote username.
• HOST - Remote hostname or IP Address.

rsync provides a number of options that control how the command behaves. The most widely used options are:

• -a, --archive, archive mode, equivalent to -rlptgoD. This option tells rsync to syncs directories recursively, transfer special and block devices, preserve symbolic links, modification times, groups, ownership, and permissions.
• -z, --compress. This option forces rsync to compresses the data as it is sent to the destination machine. Use this option only if the connection to the remote machine is slow.
• -P, equivalent to --partial --progress. When this option is used, rsync shows a progress bar during the transfer and keeps the partially transferred files. It is useful when transferring large files over slow or unstable network connections.
• --delete. When this option is used, rsync deletes extraneous files from the destination location. It is useful for mirroring.
• -q, --quiet. Use this option if you want to suppress non-error messages.
• -e. This option allows you to choose a different remote shell. By default, rsync is configured to use ssh.

Basic Rsync Usage

The most basic use case of rsync is to copy a single file from one to another local location. Here is an example:

rsync -a /opt/filename.zip /tmp/

The user running the command must have read permissions on the source location and write permissions on the destination.

Omitting the filename from the destination location copies the file with the current name. If you want to save the file under a different name, specify the new name on the destination part:

rsync -a /opt/filename.zip /tmp/newfilename.zip

The real power of rsync comes when synchronizing directories. The example below shows how to create a local backup of website files:

rsync -a /var/www/domain.com/public_html/ /var/www/domain.com/public_html_backup/

If the destination directory doesn’t exist, rsync will create it.

It is worth mentioning that rsync gives different treatment to the source directories with a trailing slash (/). If the source directory has a trailing slash, the command will copy only the directory contents to the destination directory. When the trailing slash is omitted, rsync copies the source directory inside the destination directory.

Using rsync to Sync Data from/to a remote Machine

When using rsync to transfer data remotely , it must be installed on both the source and the destination machine. The new versions of rsync are configured to use SSH as default remote shell.

In the following example, we are transferring a directory from a local to a remote machine:

rsync -a /opt/media/ remote_user@remote_host_or_ip:/opt/media/

If you haven’t set a passwordless SSH login to the remote machine, you will be asked to enter the user password.

To transfer data from a remote to a local machine, use the remote location as a source:

rsync -a remote_user@remote_host_or_ip:/opt/media/ /opt/media/

If SSH on the remote host is listening on a port other than the default 22 , specify the port using the -e option:

rsync -a -e "ssh -p 2322" /opt/media/ remote_user@remote_host_or_ip:/opt/media/

When transferring large amounts of data it is recommended to run the rsync command inside a screen session or to use the -P option:

rsync -a -P remote_user@remote_host_or_ip:/opt/media/ /opt/media/

Exclude Files and Directories

There are two options to exclude files and directories. The first option is to use the --exclude argument and specify the files and directories you want to exclude on the command line.

When excluding files or directories , you need to use their relative paths to the source location.

In the following example shows how exclude the node_modules and tmp directories:

rsync -a --exclude=node_modules --exclude=tmp /src_directory/ /dst_directory/

The second option is to use the --exclude-from option and specify the files and directories you want to exclude in a file.

rsync -a --exclude-from='/exclude-file.txt' /src_directory/ /dst_directory/

/exclude-file.txt

node_modules
tmp

[lihongjie0209]

NC

Netcat is like a swiss army knife for geeks. It can be used for just about anything involving TCP or UDP. One of its most practical uses is to transfer files. Non *nix people usually don't have SSH setup, and it is much faster to transfer stuff with netcat then setup SSH. netcat is just a single executable, and works across all platforms (Windows,Mac OS X, Linux).

On the receiving end running,

nc -l -p 1234 > out.file

will begin listening on port 1234.

On the sending end running,

nc -w 3 [destination] 1234 < out.file

will connect to the receiver and begin sending file.

For faster transfers if both sender and receiver has some basic *nix tools installed, you can compress the file during sending process,

On the receiving end,

nc -l -p 1234 | uncompress -c | tar xvfp -

On the sending end,

tar cfp - /some/dir | compress -c | nc -w 3 [destination] 1234

A much cooler but less useful use of netcat is, it can transfer an image of the whole hard drive over the wire using a command called dd.

On the sender end run,

dd if=/dev/hda3 | gzip -9 | nc -l 3333

On the receiver end,

nc [destination] 3333 | pv -b > hdImage.img.gz

Be warned that file transfers using netcat are not encrypted, anyone on the network can grab what you are sending, so use this only on trusted networks.