search

CleanCut / green

Green is a clean, colorful, fast python test runner.
MIT License
785 stars 75 forks source link

shutil cleanup sometimes fails on python 3.8 #264

Closed WolfgangFahl closed 1 year ago

WolfgangFahl commented 1 year ago

is this an upstream problem or would you be able to catch this? Our tests are marked as failures when this happens ...

Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.13/x64/bin/green", line 8, in <module>
    sys.exit(main())
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/green/cmdline.py", line 88, in main
    return _main(argv, testing)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/tempfile.py", line 827, in __exit__
    self.cleanup()
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/tempfile.py", line 831, in cleanup
    self._rmtree(self.name)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/tempfile.py", line 813, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/shutil.py", line 722, in rmtree
    onerror(os.rmdir, path, sys.exc_info())
  File "/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/shutil.py", line 720, in rmtree
    os.rmdir(path)
OSError: [Errno 39] Directory not empty: '/tmp/tmpelefosle'
CleanCut commented 1 year ago

Given that...

  1. Green is using Python's built-in tempfile handling (see snippet at bottom)
    • You can see that line 88 is simply returning, so the end of the with tempfile.TemporaryDirectory()... context is triggering a cleanup from Python's internals.
  2. No other reports of problems have arisen in the 2 years since this code has been touched

...this is most likely a problem with your project, platform, or CI setup. Perhaps you are changing permissions of a file at some point so that it cannot be deleted? Or perhaps you have some subprocess that doesn't get killed that keeps adding files even while things are getting cleaned up? If you printed a recursive directory listing of the temporary directory after hitting this error, that may shed some light on the problem.

I'm going to go ahead and close this for now, as I don't think it's Green's problem, but feel to reopen it if you discover any evidence to the contrary so we can look into it.

https://github.com/CleanCut/green/blob/4939bf8b794839ae4f45b16172fec2f6321c3d95/green/cmdline.py#L83-L88

WolfgangFahl commented 1 year ago

Or perhaps you have some subprocess that doesn't get killed that keeps adding files even while things are getting cleaned up?

Yes due to a problem in another upstream library we have processes still running and we want to ignore this for the time being. Thats's why we'd love to catch this instead of having it flagged as an error. There is no "reopen" button available.

CleanCut commented 1 year ago

There is no "reopen" button available.

My apologies! I thought that the person who created an issue would have access to the Reopen button! I guess it only shows up for me.

CleanCut commented 1 year ago

Green 3.4.3 (just released) treats the inability to clean up a temp directory that it created as a warning instead of a crash...I hope. I couldn't find a sane way to test this.

WolfgangFahl commented 1 year ago

@CleanCut excellent - since the CI picks up the latest green anyway we might see the warning sooner or later. If i spot in the logs I'll let you know.