This error message keeps appearing during the training process, but the training itself seems to be fine, i.e. the loss is decreasing. I wonder how can I get rid of this..... (btw, how can insert newline in the error message below?)
OSError: [Errno 16] Device or resource busy: '.nfs000000009886bd5100000235' File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/multiprocessing/util.py", line 186, in __call__ res = self._callback(*self._args, **self._kwargs) File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/multiprocessing/util.py", line 262, in _run_finalizers finalizer() Traceback (most recent call last): File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/shutil.py", line 480, in rmtree _rmtree_safe_fd(fd, path, onerror) File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/multiprocessing/util.py", line 186, in __call__ res = self._callback(*self._args, **self._kwargs) File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/shutil.py", line 438, in _rmtree_safe_fd onerror(os.unlink, fullname, sys.exc_info()) File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/shutil.py", line 480, in rmtree _rmtree_safe_fd(fd, path, onerror) Process Process-280: File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/shutil.py", line 436, in _rmtree_safe_fd os.unlink(name, dir_fd=topfd) File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/shutil.py", line 438, in _rmtree_safe_fd onerror(os.unlink, fullname, sys.exc_info()) File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/multiprocessing/util.py", line 262, in _run_finalizers finalizer() File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/shutil.py", line 436, in _rmtree_safe_fd os.unlink(name, dir_fd=topfd) OSError: [Errno 16] Device or resource busy: '.nfs000000009886bd5000000236' File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/multiprocessing/util.py", line 186, in __call__ res = self._callback(*self._args, **self._kwargs) OSError: [Errno 16] Device or resource busy: '.nfs000000009886bd4e00000236' File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/shutil.py", line 480, in rmtree _rmtree_safe_fd(fd, path, onerror) Process Process-278: File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/shutil.py", line 438, in _rmtree_safe_fd onerror(os.unlink, fullname, sys.exc_info()) Traceback (most recent call last): File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/shutil.py", line 436, in _rmtree_safe_fd os.unlink(name, dir_fd=topfd)
This error message keeps appearing during the training process, but the training itself seems to be fine, i.e. the loss is decreasing. I wonder how can I get rid of this..... (btw, how can insert newline in the error message below?)
OSError: [Errno 16] Device or resource busy: '.nfs000000009886bd5100000235' File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/multiprocessing/util.py", line 186, in __call__ res = self._callback(*self._args, **self._kwargs) File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/multiprocessing/util.py", line 262, in _run_finalizers finalizer() Traceback (most recent call last): File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/shutil.py", line 480, in rmtree _rmtree_safe_fd(fd, path, onerror) File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/multiprocessing/util.py", line 186, in __call__ res = self._callback(*self._args, **self._kwargs) File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/shutil.py", line 438, in _rmtree_safe_fd onerror(os.unlink, fullname, sys.exc_info()) File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/shutil.py", line 480, in rmtree _rmtree_safe_fd(fd, path, onerror) Process Process-280: File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/shutil.py", line 436, in _rmtree_safe_fd os.unlink(name, dir_fd=topfd) File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/shutil.py", line 438, in _rmtree_safe_fd onerror(os.unlink, fullname, sys.exc_info()) File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/multiprocessing/util.py", line 262, in _run_finalizers finalizer() File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/shutil.py", line 436, in _rmtree_safe_fd os.unlink(name, dir_fd=topfd) OSError: [Errno 16] Device or resource busy: '.nfs000000009886bd5000000236' File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/multiprocessing/util.py", line 186, in __call__ res = self._callback(*self._args, **self._kwargs) OSError: [Errno 16] Device or resource busy: '.nfs000000009886bd4e00000236' File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/shutil.py", line 480, in rmtree _rmtree_safe_fd(fd, path, onerror) Process Process-278: File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/shutil.py", line 438, in _rmtree_safe_fd onerror(os.unlink, fullname, sys.exc_info()) Traceback (most recent call last): File "/home/lijun/anaconda2/envs/pytorch3/lib/python3.6/shutil.py", line 436, in _rmtree_safe_fd os.unlink(name, dir_fd=topfd)