lightning-kit is an open-source reactive programming framework written in C++ designed to facilitate efficient data transfer between GPUs (Graphics Processing Units), NICs (Network Interface Cards), DPUs (Data Processing Units), and various storage devices. This project aims to optimize data transfer and processing operations, making it highly suitable for applications in high-performance computing, data analytics, and machine learning environments.
Clone the repository:
git clone https://github.com/fixstars/lightning-kit.git
cd lightning-kit
Build the project using CMake:
mkdir build && cd build
cmake ..
make
On server:
echo 2048 | sudo tee /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
echo 8 | sudo tee /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages
sudo modprobe nvidia-peermem
sudo ip link set dev <dev-name> mtu 8000
sudo ip addr add 10.0.0.2.2/24 dev <dev-name>
sudo ./build/test
On client:
sudo ip link set dev <dev-name> mtu 8000
sudo ip link addr 10.0.0.1/24 dev <dev-name>
netcat 10.0.0.2 1234 < <some-file>
We welcome contributions to lightning-kit! If you'd like to contribute, please follow these steps:
lightning-kit is released under the MIT License with common clause. See the LICENSE file for more information.