Netty 学习笔记（三）ByteBuf

上一篇文章中，我们使用 Client 发送消息给 Server，使用了 ByteBuf，本篇文章就介绍一下 ByteBuf 是什么和它的相关用法。

查看 Doc

首先我们下载源码，查看 ByteBuf 的注释。

A random and sequential accessible sequence of zero or more bytes (octets). This interface provides an abstract view for one or more primitive byte arrays (byte[]) and NIO buffers.

大意：一个长度可能为 0 的 byte 序列，支持随机和顺序访问。并且这个接口为一个或多个原始 byte 数组和 NIO 缓冲区 提供抽象视图。

PS：暂时只看这个确实无法理解 ByteBuf，让我们接着看下去

如何创建

It is recommended to create a new buffer using the helper methods in Unpooled rather than calling an individual implementation's constructor.

建议使用Unpooled中的辅助方法来创建一个新的缓冲区，而不是调用某个实现类的构造器。

示例：

ByteBuf byteBuf = Unpooled.buffer();

随机访问

用法类似原始 byte 数组，示例：

// ByteBuf 随机访问
ByteBuf byteBuf = ...;
for (int i = 0; i < byteBuf.capacity(); i++) {
    byte b = byteBuf.getByte(i);
    System.out.println((char) b);
}
// 原始 byte 数组的随机访问对比
byte[] bytes = ...;
for (int i = 0; i < bytes.length; i++) {
    byte b = bytes[i];
    System.out.println((char) b);
}

顺序访问（重点）

ByteBuf provides two pointer variables to support sequential read and write operations - readerIndex for a read operation and writerIndex for a write operation respectively.

ByteBuf 提供了两个指针变量用来支持顺序读和写操作 - 读操作的 readerIndex 和写操作的 writerIndex

readIndex与writerIndex 是使用 ByteBuf 时最重要的两个变量。贯穿了 Bytebuf 中的读/写操作。

可读字节

This segment is where the actual data is stored. Any operation whose name starts with read or skip will get or skip the data at the current readerIndex and increase it by the number of read bytes. If the argument of the read operation is also a ByteBuf and no destination index is specified, the specified buffer's writerIndex is increased together. If there's not enough content left, IndexOutOfBoundsException is raised. The default value of newly allocated, wrapped or copied buffer's readerIndex is 0.

该段存放真正的数据。任何以 read 或 skip 开头命名的操作都将获取或者跳过当前读指针（readerIndex）的数据，并将其增加读取/跳过的字节数。

当读操作的参数也是一个 ByteBuf 并且没有指定目标索引时，指定的缓冲区的写指针也会一并增长。

没有足够的内容时，调用方法将抛出 IndexOutOfBoundsException。

新分配、包装或者复制的缓冲区的读指针默认值都是 0。

示例1：迭代缓冲区可读字节

// Iterates the readable bytes of a buffer.
 ByteBuf buffer = ...;
 while (buffer.isReadable()) {
     System.out.println(buffer.readByte());
 }

示例2：读操作&读指针变化

@Test
public void readTest() {
    ByteBuf buf  = Unpooled.buffer();
    for (int i = 0; i < 10; i++) {
        buf.writeInt(i);
    }

    System.out.println("=== init ===");
    print(buf);
    System.out.println("=== skip int ===");
    buf.skipBytes(4);
    print(buf);
    for (int i = 0; i < 10; i++) {
        System.out.println("=== read:" + buf.readInt() + "===");
        print(buf);
    }

    Assert.assertEquals(buf.readerIndex(), buf.writerIndex());
    try {
        buf.readInt();
        Assert.fail();
    } catch (IndexOutOfBoundsException e) {
        System.out.println("读指针超过可读字节");
    }
}

private static void print(ByteBuf byteBuf) {
    System.out.println("readerIndex:" + byteBuf.readerIndex());
    System.out.println("writerIndex:" + byteBuf.writerIndex());
    System.out.println("capacity:" + byteBuf.capacity());
}

部分输出如下：

=== init ===
readerIndex:0
writerIndex:40
capacity:256
=== skip 4 Byte ===
readerIndex:4
writerIndex:40
capacity:256
=== read:1===
readerIndex:8
writerIndex:40
capacity:256
...
=== read:9===
readerIndex:40
writerIndex:40
capacity:256
读指针超过可读字节

可以看出，未进行读操作前，读指针一直为 0，执行 readInt 操作后，每一次增长 4（int 占 4 字节）。当没有足够的内容可读时，就会抛出 IndexOutOfBoundsException 异常。

可写字节

This segment is a undefined space which needs to be filled. Any operation whose name starts with write will write the data at the current writerIndex and increase it by the number of written bytes. If the argument of the write operation is also a ByteBuf, and no source index is specified, the specified buffer's readerIndex is increased together. If there's not enough writable bytes left, IndexOutOfBoundsException is raised. The default value of newly allocated buffer's writerIndex is 0. The default value of wrapped or copied buffer's writerIndex is the capacity of the buffer.

该段是未定义的空间，需要被数据填充。

任何以 write 开头的操作都会在当前写指针（writerIndex）处写入数据。

如果写操作的参数也是 ByteBuf，并且没有指定起始索引，则指定缓冲区的读指针（readerIndex）也会一并增长。

当没有足够的空间可写时，将抛出 IndexOutOfBoundsException 。

新分配的缓冲区的写指针为 0。

包装或者复制的缓冲区的写指针等于该缓冲区的容量（capacity）。

示例1：使用随机整数填充缓冲区

// Fills the writable bytes of a buffer with random integers.
 ByteBuf buffer = ...;
 while (buffer.maxWritableBytes() >= 4) {
     buffer.writeInt(random.nextInt());
 }

示例2：写操作&写指针变化

@Test
public void writeTest() {
    // 初始容量 10 字节，最大容量 40 字节
    ByteBuf buf = Unpooled.buffer(10, 42);
    System.out.println("=== init ===");
    print(buf);
    for (int i = 0; i < 10; i++) {
        buf.writeInt(i);
        System.out.println("=== write" + i + " ===");
        print(buf);
    }
    System.out.println("=== done ===");
    print(buf);

    try {
        System.out.println("=== 尝试超过最大容量 ===");
        buf.writeInt(1);
        Assert.fail();
    } catch (IndexOutOfBoundsException e) {
        System.out.println("没有足够的可用空间支持执行写操作！");
    }
}

部分输出如下：

=== init ===
readerIndex:0
writerIndex:0
capacity:10
=== write0 ===
readerIndex:0
writerIndex:4
capacity:10
=== write1 ===
readerIndex:0
writerIndex:8
capacity:10
=== write2 ===
readerIndex:0
writerIndex:12
capacity:42
...
=== write9 ===
readerIndex:0
writerIndex:40
capacity:42
=== done ===
readerIndex:0
writerIndex:40
capacity:42
=== 尝试超过最大容量 ===
没有足够的可用空间支持执行写操作！

每写入一个 int 数据，写指针增长 4，当空间不足时，会自动扩容（如写入 2 时，剩余空间为 2 字节，不足以写入一个 int），当达到没有足够的可写空间时，继续执行写操作，会抛出 IndexOutOfBoundsException。

可丢弃的字节

This segment contains the bytes which were read already by a read operation. Initially, the size of this segment is 0, but its size increases up to the writerIndex as read operations are executed. The read bytes can be discarded by calling discardReadBytes() to reclaim unused area as depicted by the following diagram: BEFORE discardReadBytes()

此分段的包含读操作已读取的字节。

此段最初大小为 0，但是它的大小会随着读操作的执行增加到写指针处。

已读取的字节可以通过调用方法 discardReadBytes()丢弃，这样可以回收未使用的区域。

discardReadBytes()

执行之前：

执行之后：

可读字节段的长度不变
可写字节段的长度增加，现可写字节段长度 = 容量 - 可读字节段 = 原可写字节段长度 + 原可抛弃字节段长度
实际操作是只移动了可读字节以及写指针，而没有对所有可写入字节进行擦除写。

Please note that there is no guarantee about the content of writable bytes after calling discardReadBytes(). The writable bytes will not be moved in most cases and could even be filled with completely different data depending on the underlying buffer implementation. Clearing the buffer indexes

需要注意的是：调用 discardReadBytes()方法后，无法保证可写字节的内容。在大多数情况下，可写字节不会被移动，甚至可以根据底层缓冲区实现填充完全不同的数据。

示例：

@Test
public void discardReadBytesTest() {
    ByteBuf buf = Unpooled.buffer(10, 40);
    // 第一次写入：使用 1...10 填充
    for (int i = 0; i < 10; i++) {
        buf.writeInt(i);
    }
    System.out.println("=== first fill ===");
    print(buf);
    // 重置读写指针
    buf.clear();
    System.out.println("=== reset readerIndex & writerIndex ===");
    print(buf);

    // 第二次写入：使用 5...1 填充前 20 字节
    for (int i = 5; i > 0; i--) {
        buf.writeInt(i);
    }

    System.out.println("=== second fill ===");
    print(buf);
    // 设置写指针 40
    buf.writerIndex(40);
    System.out.println("=== set writerIndex to 40 ===");
    print(buf);

    // 读取前2 个 int
    System.out.println("=== read " + buf.readInt() + " & " + buf.readInt() + " ===");
    print(buf);

    // 调用 discardReadBytes()
    buf.discardReadBytes();
    System.out.println("=== call discardReadBytes() ===");
    print(buf);

    for (int i = 0; i < 8; i++) {
        System.out.println("=== read " + buf.readInt() + " ===");
        print(buf);
    }
}

部分输出如下：

=== first fill ===
readerIndex:0
writerIndex:40
capacity:40
=== reset readerIndex & writerIndex ===
readerIndex:0
writerIndex:0
capacity:40
=== second fill ===
readerIndex:0
writerIndex:20
capacity:40
=== set writerIndex to 40 ===
readerIndex:0
writerIndex:40
capacity:40
=== read 5 & 4 ===
readerIndex:8
writerIndex:40
capacity:40
=== call discardReadBytes() ===
readerIndex:0
writerIndex:32
capacity:40
=== read 3 ===
readerIndex:4
writerIndex:32
capacity:40
...
=== read 9 ===
readerIndex:32
writerIndex:32
capacity:40

可以看到执行discardReadBytes()后，读指针 8 => 0，写指针 40 => 32。

并且如上所述discardReadBytes并没有对所有可写入字节进行擦除写。

clear()

You can set both readerIndex and writerIndex to 0 by calling clear(). It does not clear the buffer content (e.g. filling with 0) but just clears the two pointers. Please also note that the semantic of this operation is different from Buffer.clear().

你可以通过调用 clear() 方法同时将写指针和读指针设置为 0。它并不会清除缓存区中的内容（例如：使用 0 填充）仅仅只是清理读写指针。

请注意此操作的语义与Buffer.clear()不同。

执行前：

执行后：

示例：

@Test
public void clearTest() {
    ByteBuf buf = Unpooled.buffer(10, 40);
    // 写入：使用 1...10 填充
    for (int i = 0; i < 10; i++) {
        buf.writeInt(i);
    }
    System.out.println("=== fill with 1...10 ===");
    print(buf);
    // 重置读写指针
    buf.clear();
    System.out.println("=== reset readerIndex & writerIndex ===");
    print(buf);
    // 设置写指针 40
    buf.writerIndex(40);
    System.out.println("=== set writerIndex to 40 ===");
    print(buf);

    for (int i = 0; i < 10; i++) {
        System.out.println("=== read " + buf.readInt() + " ===");
        print(buf);
    }
}

部分输出如下：

=== fill with 1...10 ===
readerIndex:0
writerIndex:40
capacity:40
=== reset readerIndex & writerIndex ===
readerIndex:0
writerIndex:0
capacity:40
=== set writerIndex to 40 ===
readerIndex:0
writerIndex:40
capacity:40
=== read 0 ===
readerIndex:4
writerIndex:40
capacity:40
=== read 1 ===
readerIndex:8
writerIndex:40
capacity:40
...
=== read 9 ===
readerIndex:40
writerIndex:40
capacity:40

如上所述：clear仅仅只是将读写指针置 0，并没有清除缓存区的内容。

搜索操作

For simple single-byte searches, use indexOf(int, int, byte) and bytesBefore(int, int, byte). bytesBefore(byte) is especially useful when you deal with a NUL-terminated string. For complicated searches, use forEachByte(int, int, ByteProcessor) with a ByteProcessor implementation.

单字节搜索可以使用indexOf(int, int, byte)和bytesBefore(int, int, byte)。

其中bytesBefore(byte)在处理以 Null结尾的字符串时非常有用。

复杂搜索可以使用 ByteProcessor 实现的 forEachByte(int, int, ByteProcessor)

示例：

@Test
public void searchTest() {
    ByteBuf byteBuf = Unpooled.buffer();
    String  string  = "hello, world!";
    byteBuf.writeBytes(string.getBytes());

    // bytes of "hello, world!"
    System.out.println(Arrays.toString(string.getBytes()));

    System.out.println("=== use indexOf ===");
    System.out.println("the first 104 index:" + byteBuf.indexOf(0, byteBuf.readableBytes(), (byte) 104));
    // 反向查找
    System.out.println("the last 111 index:" + byteBuf.indexOf(byteBuf.readableBytes(), 0, (byte) 111));
    System.out.println("=== use bytesBefore ===");
    System.out.println("the first 104 index:" + byteBuf.bytesBefore(0, byteBuf.readableBytes(), (byte) 104));
    System.out.println("the first 111 index:" + byteBuf.bytesBefore(0, byteBuf.readableBytes(), (byte) 111));
}

输出如下：

[104, 101, 108, 108, 111, 44, 32, 119, 111, 114, 108, 100, 33]
=== use indexOf ===
the first 104 index:0
the last 111 index:8
=== use bytesBefore ===
the first 104 index:0
the first 111 index:4

indexOf 与 bytesBefore用法稍有不同。

indexOf(fromIndex, toIndex, byte value)：在缓冲区的fromIndex到toIndex区间定位第一个目标值。如果fromIndex小于toIndex则将反向查找。
bytesBefore(index, length, byte value)：在缓冲区的index到index+length区间定位第一个目标值。

注意：两者均不会修改缓冲区的读写指针

标记并重置

There are two marker indexes i n every buffer. One is for storing readerIndex and the other is for storing writerIndex. You can always reposition one of the two indexes by calling a reset method. It works in a similar fashion to the mark and reset methods in InputStream except that there's no readlimit.

每个缓冲区中有两个标记索引。一个用于存储 readerIndex ，另一个用于存储writerIndex。您始终可以通过调用reset方法重新定位两个索引中的一个。它的工作方式与 InputStream 中的mark和reset方法类似，只是没有readlimit。

示例：

@Test
public void markAndResetTest() {
    ByteBuf buf = Unpooled.buffer();
    buf.writerIndex(40);
    System.out.println("=== init buf ===");
    print(buf);

    buf.readerIndex(40);
    System.out.println("=== set readerIndex to 40 ===");
    print(buf);
    // reset readerIndex
    System.out.println("=== reset readerIndex ===");
    buf.resetReaderIndex();
    print(buf);
    assertEquals(0, buf.readerIndex());

    buf.readerIndex(10);
    buf.markReaderIndex();
    System.out.println("=== set readerIndex to 10 and mark it ===");
    print(buf);

    buf.readerIndex(40);
    buf.resetReaderIndex();
    System.out.println("=== set readerIndex to 40 and reset it ===");
    print(buf);
    assertEquals(10, buf.readerIndex());

}

输出如下：

=== init buf ===
readerIndex:0
writerIndex:40
capacity:256
=== set readerIndex to 40 ===
readerIndex:40
writerIndex:40
capacity:256
=== reset readerIndex ===
readerIndex:0
writerIndex:40
capacity:256
=== set readerIndex to 10 and mark it ===
readerIndex:10
writerIndex:40
capacity:256
=== set readerIndex to 40 and reset it ===
readerIndex:10
writerIndex:40
capacity:256

第一次重置读指针时，没有标记读指针，因此重置后，读指针 40 => 0。

第二次重置读指针时，由于读指针等于 10 时，标记了读指针，因此重置后，读指针 40 => 10。

派生缓冲区

You can create a view of an existing buffer by calling one of the following methods: duplicate() slice() slice(int, int) readSlice(int) retainedDuplicate() retainedSlice() retainedSlice(int, int) readRetainedSlice(int) A derived buffer will have an independent readerIndex, writerIndex and marker indexes, while it shares other internal data representation, just like a NIO buffer does. In case a completely fresh copy of an existing buffer is required, please call copy() method instead. Non-retained and retained derived buffers Note that the duplicate(), slice(), slice(int, int) and readSlice(int) does NOT call retain() on the returned derived buffer, and thus its reference count will NOT be increased. If you need to create a derived buffer with increased reference count, consider using retainedDuplicate(), retainedSlice(), retainedSlice(int, int) and readRetainedSlice(int) which may return a buffer implementation that produces less garbage. Conversion to existing JDK types

可以通过调用以下方法之一来创建现有缓冲区的视图：

duplicate()
slice()
slice(int，int)
readSlice(INT)
retainedDuplicate()
retainedSlice()
retainedSlice(int，int)
readRetainedSlice(int) 派生缓冲区将具有独立的读索引、写索引和标记索引，而与缓冲区共享其他内部数据表示，就像NIO缓冲区一样。（浅拷贝）如果需要现有缓冲区的全新副本，请改为调用copy()方法。（深拷贝）

示例：
```
@Test
public void derivedBufferTest() {
  ByteBuf byteBuf = Unpooled.buffer();

  byteBuf.writeBytes("Hello, world!".getBytes(CharsetUtil.UTF_8));

  byteBuf.readByte();
  ByteBuf duplicate = byteBuf.duplicate();
  ByteBuf copy      = byteBuf.copy();

  byteBuf.setByte(3, 'J');
  System.out.println(duplicate.toString(CharsetUtil.UTF_8));
  System.out.println(copy.toString(CharsetUtil.UTF_8));

  System.out.println("=== byteBuf ===");
  print(byteBuf);
  System.out.println("=== duplicate ===");
  print(duplicate);
  System.out.println("=== copy ===");
  print(copy);

  System.out.println("=== byteBuf call readByte() ===");
  byteBuf.readByte();
  System.out.println("=== byteBuf ===");
  print(byteBuf);
  System.out.println("=== duplicate ===");
  print(duplicate);
  System.out.println("=== copy ===");
  print(copy);
}
```
输出如下：
```
elJo, world!
ello, world!
=== byteBuf ===
readerIndex:1
writerIndex:13
capacity:256
=== duplicate ===
readerIndex:1
writerIndex:13
capacity:256
=== copy ===
readerIndex:0
writerIndex:12
capacity:12
=== byteBuf call readByte() ===
=== byteBuf ===
readerIndex:2
writerIndex:13
capacity:256
=== duplicate ===
readerIndex:1
writerIndex:13
capacity:256
=== copy ===
readerIndex:0
writerIndex:12
capacity:12
```
从输出可以看出：
- copy 属于深拷贝，拥有独立的数据空间以及独立的读写索引。拷贝时，读写索引置 0。
- duplicate 属于浅拷贝，拥有缓冲区相同的数据空间，以及独立的读写索引。拷贝时，读写索引初始值等于当前缓冲区读写索引值。
非保留和保留的派生缓冲区

Note that the duplicate(), slice(), slice(int, int) and readSlice(int) does NOT call retain() on the returned derived buffer, and thus its reference count will NOT be increased. If you need to create a derived buffer with increased reference count, consider using retainedDuplicate(), retainedSlice(), retainedSlice(int, int) and readRetainedSlice(int) which may return a buffer implementation that produces less garbage. Conversion to existing JDK types

注意：duplicate()，slice()，slice(int，int)和readSlice(int)不会对返回的派生缓冲区调用retain()，因此不会增加其引用计数。如果需要创建具有增加引用计数的派生缓冲区，请考虑使用retainDuplicate()，retainedSlice()，retainedSlice(int，int)和readRetainedSlice(int)，这可能会返回产生较少垃圾的缓冲区实现。

转换为现有JDK类型

字节数组（Byte array）

If a ByteBuf is backed by a byte array (i.e. byte[]), you can access it directly via the array() method. To determine if a buffer is backed by a byte array, hasArray() should be used.

如果 ByteBuf 由字节数组（即byte[]）支持，则可以通过array()方法直接访问它。要确定缓冲区是否由字节数组支持，应使用hasArray()。

示例：

@Test
public void byteArrayTest() {
    ByteBuf byteBuf = Unpooled.buffer();
    byte[]  bytes   = "Hello, world!".getBytes(CharsetUtil.UTF_8);
    byteBuf.writeBytes(bytes);

    // 调用 array() 前，需要检测是否支持字符数组
    Assert.assertTrue(byteBuf.hasArray());
    byte[] bytes2 = byteBuf.array();
    for (int i = 0; i < bytes.length; i++) {
        Assert.assertEquals(bytes[i], bytes2[i]);
    }
}

非阻塞缓冲区（NIO Buffers）

If a ByteBuf can be converted into an NIO ByteBuffer which shares its content (i.e. view buffer), you can get it via the nioBuffer() method. To determine if a buffer can be converted into an NIO buffer, use nioBufferCount().

如果可以将ByteBuf转换为共享其内容的 NIO ByteBuffer（即视图缓冲区），则可以通过nioBuffer()方法获取它。要确定缓冲区是否可以转换为NIO缓冲区，请使用nioBufferCount()。

字符串

Various toString(Charset) methods convert a ByteBuf into a String. Please note that toString() is not a conversion method.

各种toString(Charset)方法将ByteBuf转换为 String。注意，toString()不是转换方法。

示例：

@Test
public void stringsTest() {
    ByteBuf byteBuf = Unpooled.buffer();
    byteBuf.writeBytes("Hello, world!".getBytes());

    System.out.println("=== toString() ===");
    System.out.println(byteBuf.toString());
    System.out.println("=== toString(Charset) ===");
    System.out.println(byteBuf.toString(CharsetUtil.UTF_8));
}

输出如下：

=== toString() ===
UnpooledByteBufAllocator$InstrumentedUnpooledUnsafeHeapByteBuf(ridx: 0, widx: 13, cap: 256)
=== toString(Charset) ===
Hello, world!

I/O 流（I/O Streams）

Please refer to ByteBufInputStream and ByteBufOutputStream.

参考 ByteBufInputStream 和 ByteBufOutputStream。

xiwenAndlejian / my-blog