search

zilliztech / milvus-migration

20 stars 6 forks source link

[Enhancement]: improve copy performance with Kernel TLS #61

Open huanghaoyuanhhy opened 1 year ago

huanghaoyuanhhy commented 1 year ago

What would you like to be added?

In cross-cloud or cross-region data copying, the data needs to be read from the source socket and then written to the destination socket, causing two user-kernel mode switches. We can use the splice system call to avoid the overhead of context switching, where the kernel directly copies the data. The io.Copy function in Golang automatically utilizes this optimization. However, due to the presence of TLS encryption, data must be encrypted and decrypted in user mode. In newer Linux kernels (≥ 4.17), Kernel TLS (kTLS) feature has been implemented, which can be leveraged to achieve the copying functionality.

Why is this needed?

Can significantly improve performance.

Anything else?

kernel doc nginx blog

xiaofan-luan commented 1 year ago

any more description?

huanghaoyuanhhy commented 1 year ago

@xiaofan-luan Sorry for busy work. More explanations have been added.