Dpf blocks result file - Githubissues

janvonrickenbach commented 5 months ago

Before submitting the issue

[X] I have checked for Compatibility issues
[X] I have searched among the existing issues
[X] I am using a Python virtual environment

Description of the bug

It looks like dpf blocks the rst file even if it is not actively reading it. Consider the following snipped (use whatever rst file).

import pathlib
import shutil
import tempfile
from ansys.dpf.core import start_local_server, Model

server = start_local_server()
tempdir = tempfile.TemporaryDirectory()
shutil.copy( "file.rst", tempdir.name)

model = Model(str(pathlib.Path(tempdir.name) / "file.rst"))

model.results

The snippet copies the rst file in temp folder and on exit it tries to delete the temp folder. This fails with the error below [1]. I observed similar behaviour when I used the python result in Mechanical and run "Clear Generated Data". Mechanical failed to move the rst file because apparently dpf still had it open. I think DPF should "release" the rst file after its operators are run.

[1] Traceback (most recent call last): File "C:\Users\jvonrick\AppData\Local\Programs\Python\Python310\lib\shutil.py", line 617, in _rmtree_unsafe os.unlink(fullname) PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\Users\jvonrick\AppData\Local\Temp\tmp1s6_bcm4\file.rst'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\jvonrick\AppData\Local\Programs\Python\Python310\lib\tempfile.py", line 843, in onerror _os.unlink(path) PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\Users\jvonrick\AppData\Local\Temp\tmp1s6_bcm4\file.rst'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "C:\Users\jvonrick\AppData\Local\Programs\Python\Python310\lib\weakref.py", line 667, in _exitfunc f() File "C:\Users\jvonrick\AppData\Local\Programs\Python\Python310\lib\weakref.py", line 591, in call return info.func(*info.args, **(info.kwargs or {})) File "C:\Users\jvonrick\AppData\Local\Programs\Python\Python310\lib\tempfile.py", line 859, in _cleanup cls._rmtree(name, ignore_errors=ignore_errors) File "C:\Users\jvonrick\AppData\Local\Programs\Python\Python310\lib\tempfile.py", line 855, in _rmtree _shutil.rmtree(name, onerror=onerror) File "C:\Users\jvonrick\AppData\Local\Programs\Python\Python310\lib\shutil.py", line 749, in rmtree return _rmtree_unsafe(path, onerror) File "C:\Users\jvonrick\AppData\Local\Programs\Python\Python310\lib\shutil.py", line 619, in _rmtree_unsafe onerror(os.unlink, fullname, sys.exc_info()) File "C:\Users\jvonrick\AppData\Local\Programs\Python\Python310\lib\tempfile.py", line 846, in onerror cls._rmtree(path, ignore_errors=ignore_errors) File "C:\Users\jvonrick\AppData\Local\Programs\Python\Python310\lib\tempfile.py", line 855, in _rmtree _shutil.rmtree(name, onerror=onerror) File "C:\Users\jvonrick\AppData\Local\Programs\Python\Python310\lib\shutil.py", line 749, in rmtree return _rmtree_unsafe(path, onerror) File "C:\Users\jvonrick\AppData\Local\Programs\Python\Python310\lib\shutil.py", line 600, in _rmtree_unsafe onerror(os.scandir, path, sys.exc_info()) File "C:\Users\jvonrick\AppData\Local\Programs\Python\Python310\lib\shutil.py", line 597, in _rmtree_unsafe with os.scandir(path) as scandir_it: NotADirectoryError: [WinError 267] The directory name is invalid: 'C:\Users\jvonrick\AppData\Local\Temp\tmp1s6_bcm4\file.rst'

Steps To Reproduce

See above

Which Operating System causes the issue?

Windows

Which DPF/Ansys version are you using?

DPF Server 2024.2.pre0

Which Python version causes the issue?

3.10

Installed packages

Package Version Editable project location

accessible-pygments 0.0.4
aiohttp 3.9.1
aiosignal 1.3.1
alabaster 0.7.16
ansys-acp-core 0.0.dev0 D:\ANSYSDev\pyacp-private ansys-api-acp 0.1.0.dev3 ansys-api-mapdl 0.5.1 ansys-api-mechanical 0.1.1 ansys-api-platform-instancemanagement 1.0.0 ansys-api-tools-filetransfer 0.1.dev5 ansys-api-utilities-filetransfer 0.1.dev0 ansys-dpf-composites 0.3.2 ansys-dpf-core 0.10.1 ansys-mapdl-core 0.67.0 ansys-mapdl-reader 0.53.0 ansys-math-core 0.1.3 ansys-mechanical-core 0.10.5 ansys-mechanical-env 0.1.0 ansys-platform-instancemanagement 1.1.2 ansys-pythonnet 3.1.0rc3 ansys-sphinx-theme 0.13.0 ansys-tools-filetransfer 0.1.dev0 ansys-tools-local-product-launcher 0.1.dev0 ansys-tools-path 0.3.2 ansys-utilities-filetransfer 0.1.dev0 anyio 4.2.0 appdirs 1.4.4 argon2-cffi 23.1.0 argon2-cffi-bindings 21.2.0 arrow 1.3.0 asttokens 2.4.1 async-lru 2.0.4 async-timeout 4.0.3 attrs 23.2.0 Babel 2.14.0 backports.entry-points-selectable 1.3.0 beautifulsoup4 4.12.2 black 23.3.0 bleach 6.1.0 bokeh 3.3.1 cachetools 5.3.2 certifi 2023.11.17 cffi 1.16.0 cfgv 3.4.0 charset-normalizer 3.3.2 click 8.1.7 clr-loader 0.2.6 colorama 0.4.6 comm 0.2.1 contourpy 1.2.0 coverage 7.4.0 cycler 0.12.1 debugpy 1.8.0 decopatch 1.4.10 decorator 5.1.1 defusedxml 0.7.1 distlib 0.3.8 docker 7.0.0 docutils 0.19 exceptiongroup 1.2.0 executing 2.0.1 fastjsonschema 2.19.1 filelock 3.13.1 fonttools 4.47.2 fqdn 1.5.1 frozenlist 1.4.1 geomdl 5.3.1 google-api-core 2.15.0 google-api-python-client 2.113.0 google-auth 2.26.1 google-auth-httplib2 0.2.0 googleapis-common-protos 1.62.0 grpcio 1.60.0 grpcio-health-checking 1.48.2 httplib2 0.22.0 hypothesis 6.92.6 identify 2.5.33 idna 3.6 imageio 2.28.1 imagesize 1.4.1 importlib-metadata 7.0.1 iniconfig 2.0.0 ipycanvas 0.13.1 ipydatawidgets 4.3.5 ipyevents 2.0.2 ipykernel 6.28.0 ipython 8.18.1 ipython-genutils 0.2.0 ipytree 0.2.2 ipyvtklink 0.2.3 ipywidgets 8.1.1 isoduration 20.11.0 jedi 0.19.1 Jinja2 3.1.2 json5 0.9.14 jsonpointer 2.4 jsonschema 4.20.0 jsonschema-specifications 2023.12.1 jupyter_client 8.6.0 jupyter_core 5.7.1 jupyter-events 0.9.0 jupyter-lsp 2.2.0 jupyter_server 2.12.3 jupyter_server_proxy 4.1.0 jupyter_server_terminals 0.5.1 jupyterlab 4.0.8 jupyterlab_pygments 0.3.0 jupyterlab_server 2.25.1 jupyterlab-widgets 3.0.9 kiwisolver 1.4.5 linkify-it-py 2.0.2 makefun 1.15.2 Markdown 3.5.1 markdown-it-py 3.0.0 MarkupSafe 2.1.3 matplotlib 3.8.2 matplotlib-inline 0.1.6 mdit-py-plugins 0.4.0 mdurl 0.1.2 mistune 3.0.2 mock 5.1.0 more-itertools 10.2.0 multidict 6.0.4 mypy 1.8.0 mypy-extensions 1.0.0 nbclient 0.9.0 nbconvert 7.14.0 nbformat 5.9.2 nest-asyncio 1.5.8 nodeenv 1.8.0 notebook 7.0.6 notebook_shim 0.2.3 numpy 1.26.3 numpydoc 1.6.0 overrides 7.4.0 packaging 23.2 pandas 2.1.3 pandocfilters 1.5.0 panel 1.3.1 param 2.0.1 parso 0.8.3 Paste 3.5.3 pathspec 0.12.1 pillow 10.2.0 pip 22.3.1 platformdirs 4.1.0 pluggy 1.3.0 pooch 1.8.0 pre-commit 3.6.0 prometheus-client 0.19.0 prompt-toolkit 3.0.43 protobuf 3.20.3 protoc-gen-swagger 0.1.0 psutil 5.9.7 pure-eval 0.2.2 py-cpuinfo 9.0.0 pyansys-tools-versioning 0.5.0 pyasn1 0.5.1 pyasn1-modules 0.3.0 pycparser 2.21 pydata-sphinx-theme 0.15.1 Pygments 2.17.2 pyiges 0.3.1 pypandoc 1.12 pyparsing 3.1.1 pytest 7.4.4 pytest-benchmark 4.0.0 pytest-cases 3.8.1 pytest-cov 4.1.0 python-dateutil 2.8.2 python-json-logger 2.0.7 pythreejs 2.4.2 pytz 2023.3.post1 pyvista 0.43.2 pyviz_comms 3.0.0 pywin32 306 pywinpty 2.0.12 PyYAML 6.0.1 pyzmq 25.1.2 readable 0.0.post30 referencing 0.32.1 requests 2.31.0 rfc3339-validator 0.1.4 rfc3986-validator 0.1.1 rpds-py 0.16.2 rsa 4.9 scipy 1.11.4 scooby 0.9.2 Send2Trash 1.8.2 setuptools 69.0.3 setuptools-scm 6.4.2 shortuuid 1.0.11 simpervisor 1.0.0 six 1.16.0 sniffio 1.3.0 snowballstemmer 2.2.0 sortedcontainers 2.4.0 soupsieve 2.5 Sphinx 6.2.1 sphinx-autodoc-typehints 1.23.0 sphinx-copybutton 0.5.2 sphinx_design 0.5.0 sphinx-gallery 0.13.0 sphinxcontrib-applehelp 1.0.7 sphinxcontrib-devhelp 1.0.5 sphinxcontrib-htmlhelp 2.0.4 sphinxcontrib-jsmath 1.0.1 sphinxcontrib-qthelp 1.0.6 sphinxcontrib-serializinghtml 1.1.9 stack-data 0.6.3 tabulate 0.9.0 Tempita 0.5.2 terminado 0.18.0 tinycss2 1.2.1 toml 0.10.2 tomli 2.0.1 tornado 6.4 tqdm 4.66.1 traitlets 5.14.1 traittypes 0.2.1 trame 3.5.0 trame-client 2.14.2 trame-server 2.15.0 trame-vtk 2.6.3 trame-vuetify 2.3.1 types-protobuf 4.24.0.20240106 types-python-dateutil 2.8.19.20240106 typing_extensions 4.9.0 tzdata 2023.3 uc-micro-py 1.0.2 uri-template 1.3.0 uritemplate 4.1.1 urllib3 2.1.0 virtualenv 20.25.0 vtk 9.3.0 waitress 2.1.2 wcwidth 0.2.13 webcolors 1.13 webencodings 0.5.1 WebError 0.13.1 WebOb 1.8.7 websocket-client 1.7.0 WebTest 3.0.0 wheel 0.37.1 widgetsnbextension 4.0.9 wslink 1.12.4 xyzservices 2023.10.1 yarl 1.9.4 zipp 3.17.0 zope.component 6.0 zope.event 5.0 zope.hookable 5.4 zope.interface 6.0

cbellot000 commented 5 months ago

Hi @janvonrickenbach, indeed, the model opens a streams to read the file (to not reopen the file at each request and to keep some data cached). you can release the file by calling: model.release_streams()

greschd commented 5 months ago

@cbellot000 does this happen automatically when the Model goes out of scope and / or Python exits? If not, I think we should add it 🙂

janvonrickenbach commented 5 months ago

As discussed: DPF closes the stream when the model goes out of scope. The problem is that the order in which the stream is closed and the tempfile is removed is undefined (both happen at the end of the script). A possible solution is to set the dpf model to None before the end of the script. This is the approach we will follow in pydpf-composites.

greschd commented 5 months ago

A possible solution is to set the dpf model to None before the end of the script.

Wouldn't an explicit release_streams be better? Otherwise, if another object holds a reference to the DPF model we will get the problem again.

ansys / pydpf-core

Dpf blocks result file #1373