search

kig / rdma-pipe

Utility programs to pipe data across a RDMA-capable network
Apache License 2.0
8 stars 3 forks source link

Library bindings to use the fast I/O in Python #5

Open kig opened 11 months ago

kig commented 11 months ago

Doing file I/O at 20 GB/s would be quite nice for loading large AI models.

Could we create library bindings for the file I/O and the RDMA transport separately, so that you can read and write files at high speed, in addition to sending them across the network.