uqfoundation / pyina

MPI parallel map and cluster scheduling
http://pyina.rtfd.io
Other
61 stars 8 forks source link

pypy doesn't cleanup tmp files #47

Closed mmckerns closed 1 year ago

mmckerns commented 2 years ago

When the test suite is run, pypy doesn't clean up the tmp files used in passing information.

For example:

tmp101fuhef.pik  tmpc9wd7ike.py   tmpnhkw0qwg.py
tmp4wnottod.arg  tmpf829_88z.arg  tmptj8jv72n.py
tmp7cfazakp.py   tmpgzhcrjgs.pik  tmpu2lp6eya.arg
tmp7fi5f4c3.pik  tmpj6qtdeb2.arg  tmpurg6xn4t.arg
tmp7qqcyv5s.py   tmpjvmdrf02.arg  tmpxmamvqhx.arg
tmp8aurdvkm.arg  tmpk3a29ryq.py   tmpzd59ylu_.arg
tmp9_3r7057.arg  tmpl2t3r4mc.arg
tmpakawfcw_.pik  tmpm27ve3j1.pik
tmpato9fs89.pik  tmpmxd90ax9.arg

are left after running test_ezmap.py

mmckerns commented 1 year ago

it appears that this is due to pypy not deleting temporary files when the instance is deleted. dill.temp.dump creates a tempfile.NamedTemporaryFile that is configured to be deleted when the instance is deleted (i.e. delete=True).

Python 3.9.15 (main, Oct 12 2022, 04:43:41) 
[Clang 10.0.1 (clang-1001.0.46.4)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import dill
>>> f = dill.temp.dump(lambda x:x, dir='.')
>>> s = f.name
>>> del f
>>> f = open(s)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
FileNotFoundError: [Errno 2] No such file or directory: '/Users/mmckerns/tmph0fkky_i'
>>>
Python 3.9.12 (05fbe3aa5b0845e6c37239768aa455451aa5faba, Mar 29 2022, 09:54:47)
[PyPy 7.3.9 with GCC Apple LLVM 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>> import dill
>>>> f = dill.temp.dump(lambda x:x, dir='.')
>>>> s = f.name
>>>> del f
>>>> f = open(s) 
>>>> dill.temp.load(f)
<function <lambda> at 0x000000010a060340>

However, it appears that calling f.close() will delete the file.