更省内存的Server以及进程启动

[eedalong]

我们涉及到如下几个内存类型：

• [ ] Shared Memory
• [ ] Pinned Memory
• [ ] IB Registered Memory

需要在Load数据的时候进行处理使得他们三个是同一个内存，避免内存峰值过高。 @joker-xii

[eedalong]

When doing initilization, we need to load data in a shared memory and pinned to GPU. So this will be done in 3 steps:

1. Load a pytorch tensor directly to a shared memory.
2. Create a pytorch tensor from this shared memory
3. Create ShardTensor & DistTensorPGAS

[eedalong]

300G feature.pt

tensor = torch.load("feature.pt")

sh_mem_tensor = quiver_feature.load("feature.pt")

• [ ] open file: ptr = shm_open("feature.pt", offset = sizeof(meta))
• [ ] create tensor: tensor = torch.from_blob(ptr, options)
• [ ] set attributes: tensor.shm_memory = True

[meta_data, tensor_data]

meta = torch.load_meta("feature.pt")

tensor.share_mem() register_to_device() shard_tensor = ShardTensor(tensor) ib_register(tensor)