Sometimes, after successful copying of a dataset - at least from one mount point to another - the process fails, because shutils.rmtree complains about a non-empty tmp folder:
2022-11-28 20:58:20,429 dropboxhandler line:274 (levelname)-8s An error occured while handeling file. Creating error marker file /mnt/nfs/qbic/dropboxes/qeana03_imgagdna/incoming/MARKER_error_20221127232052_[...]_nanopore, remove if you fixed the error. Error was: Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/dropboxhandler-1.8.1-py2.6.egg/dropboxhandler/dropboxhandler.py", line 248, in _handle_file self.to_openbis(origin, file, perms=perms) File "/usr/lib/python2.6/site-packages/dropboxhandler-1.8.1-py2.6.egg/dropboxhandler/dropboxhandler.py", line 163, in to_openbis file, dest_file, tmpdir=self._tmpdir, perms=perms File "/usr/lib/python2.6/site-packages/dropboxhandler-1.8.1-py2.6.egg/dropboxhandler/fstools.py", line 249, in recursive_copy shutil.rmtree(tmpdir) File "/usr/lib64/python2.6/shutil.py", line 212, in rmtree rmtree(fullname, ignore_errors, onerror) File "/usr/lib64/python2.6/shutil.py", line 212, in rmtree rmtree(fullname, ignore_errors, onerror) File "/usr/lib64/python2.6/shutil.py", line 212, in rmtree rmtree(fullname, ignore_errors, onerror) File "/usr/lib64/python2.6/shutil.py", line 221, in rmtree onerror(os.rmdir, path, sys.exc_info()) File "/usr/lib64/python2.6/shutil.py", line 219, in rmtree os.rmdir(path) OSError: [Errno 39] Directory not empty: '/mnt/nfs/qbic/dropboxes/tmp/tmp6ENlDz/[...]_nanopore/20221122_[...]/fast5_pass'
While the data is complete, the process has to be repeated, as the checksums were not created due to the exception. It is likely that setting ignore_errors=True in the following call would fix the problem:
Sometimes, after successful copying of a dataset - at least from one mount point to another - the process fails, because shutils.rmtree complains about a non-empty tmp folder:
2022-11-28 20:58:20,429 dropboxhandler line:274 (levelname)-8s An error occured while handeling file. Creating error marker file /mnt/nfs/qbic/dropboxes/qeana03_imgagdna/incoming/MARKER_error_20221127232052_[...]_nanopore, remove if you fixed the error. Error was: Traceback (most recent call last): File "/usr/lib/python2.6/site-packages/dropboxhandler-1.8.1-py2.6.egg/dropboxhandler/dropboxhandler.py", line 248, in _handle_file self.to_openbis(origin, file, perms=perms) File "/usr/lib/python2.6/site-packages/dropboxhandler-1.8.1-py2.6.egg/dropboxhandler/dropboxhandler.py", line 163, in to_openbis file, dest_file, tmpdir=self._tmpdir, perms=perms File "/usr/lib/python2.6/site-packages/dropboxhandler-1.8.1-py2.6.egg/dropboxhandler/fstools.py", line 249, in recursive_copy shutil.rmtree(tmpdir) File "/usr/lib64/python2.6/shutil.py", line 212, in rmtree rmtree(fullname, ignore_errors, onerror) File "/usr/lib64/python2.6/shutil.py", line 212, in rmtree rmtree(fullname, ignore_errors, onerror) File "/usr/lib64/python2.6/shutil.py", line 212, in rmtree rmtree(fullname, ignore_errors, onerror) File "/usr/lib64/python2.6/shutil.py", line 221, in rmtree onerror(os.rmdir, path, sys.exc_info()) File "/usr/lib64/python2.6/shutil.py", line 219, in rmtree os.rmdir(path) OSError: [Errno 39] Directory not empty: '/mnt/nfs/qbic/dropboxes/tmp/tmp6ENlDz/[...]_nanopore/20221122_[...]/fast5_pass'While the data is complete, the process has to be repeated, as the checksums were not created due to the exception. It is likely that setting ignore_errors=True in the following call would fix the problem:
https://github.com/qbicsoftware/dropboxhandler/blob/7c44c7f750c12ebed8d26bef9f54a0c4f71d6c59/dropboxhandler/fstools.py#L249
See documentation: https://docs.python.org/3/library/shutil.html#shutil.rmtree