ray-project / ray_lightning

Pytorch Lightning Distributed Accelerators using Ray
Apache License 2.0
211 stars 34 forks source link

warning from the horovod trainer #168

Closed JiahaoYao closed 2 years ago

JiahaoYao commented 2 years ago
(BaseHorovodWorker pid=4592) [W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
(BaseHorovodWorker pid=452, ip=172.31.46.122) [W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
(BaseHorovodWorker pid=4592) [W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
(BaseHorovodWorker pid=452, ip=172.31.46.122) [W pthreadpool-cpp.cc:90] Warning: Leaking Caffe2 thread-pool after fork. (function pthreadpool)
/home/ray/anaconda3/lib/python3.8/site-packages/pytorch_lightning/core/datamodule.py:126: LightningDeprecationWarning: DataModule property `test_transforms` was deprecated in v1.5 and will be removed in v1.7.
  rank_zero_deprecation(
/home/ray/anaconda3/lib/python3.8/site-packages/torchvision/datasets/mnist.py:498: UserWarning: The given NumPy array is not writeable, and PyTorch does not support non-writeable tensors. This means you can write to the underlying (supposedly non-writeable) NumPy array using the tensor. You may want to copy the array to protect its data or make it writeable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  /pytorch/torch/csrc/utils/tensor_numpy.cpp:180.)
  return torch.from_numpy(parsed.astype(m[2], copy=False)).view(*s)
JiahaoYao commented 2 years ago

tag: https://github.com/pytorch/pytorch/issues/57273 https://discuss.pytorch.org/t/userwarning-the-given-numpy-array-is-not-writeable/78748

wonder whether this is due to pytorch

JiahaoYao commented 2 years ago

tag: https://discuss.pytorch.org/t/warning-leaking-caffe2-thread-pool-after-fork-function-pthreadpool/127559

JiahaoYao commented 2 years ago

fixed by the latest pytorch version