search

usgs / groundmotion-processing

Parsing and processing ground motion data
Other
54 stars 42 forks source link

'assemble' fails when encountering .DS_Store files during shutil tmp directory creation and deletion #899

Closed gferragu closed 2 years ago

gferragu commented 2 years ago

Describe the bug When the auto-generated .DS_Store files are created by Mac's Finder in a gmprocess project's subdirectories (data/raw/ for example), subcommand assemble may fail due to an error encountered in read_directory.py This is likely due to .DS_Store file permissions, which results in a non-empty directory during the call to shutil.rmtree() here

Adding .DS_Store to the EXT_IGNORE list does not fix the issue to the how ignored files are checked here.

A fix and possible improvement to the existing method of looping over files and checking their extensions could be to use the built-in shutil.ignore_patterns() function, which can be passed as a parameter to the initial call of shutil.copytree() here so that unwanted files (both .DS_Store and those with extensions listed in EXT_IGNORE) are never copied in the first place.

emthompson-usgs commented 2 years ago

I don't really follow what is happening. I don't think that it should matter if .DS_Store is empty since shutil.rmtree removes a directory tree, and there are already many non empty directories in the path handed to it. If it is a permissions issue in that your user cannot delete the directory, that doesn't seem like a gmprocess issue.

emthompson-usgs commented 2 years ago

Closd by #901.