search

hpcaitech / TensorNVMe

A Python library transfers PyTorch tensors between CPU and NVMe
93 stars 18 forks source link

Will this kill the page cache? #42

Open marsupialtail opened 2 years ago

marsupialtail commented 2 years ago

I see that this library is not using DirectIO to flush out the weights to disk:

this->fd = open(filename.c_str(), O_RDWR | O_CREAT, S_IRUSR | S_IWUSR);

Will this kill the page cache if you have a lot of weights?

ver217 commented 2 years ago

Yes. We will add a flag to control it in the next release.