search

HumanCellAtlas / dcp-cli

DEPRECATED - HCA Data Coordination Platform Command Line Interface
https://hca.readthedocs.io/
MIT License
6 stars 8 forks source link

Fall back to copying when os.link raises error #521

Closed chmreid closed 4 years ago

chmreid commented 4 years ago

This addresses a permission denied error on NFS file systems that is caused by the os.link call. Addresses #519

Previously we were catching a "too many links" exception, now we are also catching a "permission denied" exception and an "operation not permitted" exception. (Note that we may be able to remove the "operation not permitted" exception number from the list of exceptions that lead us to fall back to copying.)

Useful Links

Useful discussion of possible approaches to solving this better: https://bugs.python.org/issue37612

Useful info about NFS and Python's os module: https://web.archive.org/web/20100912144722/http://www.unixcoding.org/NFSCoding

Note: this PR does not (yet) have tests for the new error codes being caught/handled. Tests TBA.

jessebrennan commented 4 years ago

@chmreid I see that you requested my review, but didn't assign the PR to me. I'm confused by this. Did you see this Slack thread? https://ucsc-cgl.slack.com/archives/C705Y6G9Z/p1585093339100200

chmreid commented 4 years ago

@jessebrennan The problem is with the master branch, nothing specific to this PR. I tried re-running Travis tests for master and they failed: https://travis-ci.org/github/HumanCellAtlas/dcp-cli/builds/654957449

We can pick up on this PR once master is passing.

chmreid commented 4 years ago

@jessebrennan Tests are passing, can you give this a review now?

hannes-ucsc commented 4 years ago

I would like the opportunity to review this. I'm currently not assigned so I'll hold off for now.

chmreid commented 4 years ago

@hannes-ucsc I have assigned this issue to you, please review when you get the chance. Thanks!

chmreid commented 4 years ago

@hannes-ucsc Thank you for the comments. I have addressed your feedback, and this PR is ready for another review. Would you please review it at your convenience? Thanks.