Permission Error [WinError 32]: dagster_webserver tempfile process conflict

[TobiSan5]

Dagster version

dagster, version 1.6.11

What's the issue?

I just installed dagster and dagit to a project environment (using conda/conda-forge) based on python 3.11. When I tried to start the webserver with the dagit script on the command line, the following error occured.

Traceback (most recent call last):
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\tempfile.py", line 893, in onerror
    _os.unlink(path)
PermissionError: [WinError 5] Ingen tilgang: 'C:\\Dropbox\\CODE2\\xlwings\\excel-ics\\tmpka29b0dd\\schedules'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\ProgramData\Miniconda3\envs\xlwings\Scripts\dagit-script.py", line 9, in <module>
    sys.exit(main())
             ^^^^^^
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\site-packages\dagster_webserver\cli.py", line 316, in main
    cli(auto_envvar_prefix="DAGSTER_WEBSERVER")
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\site-packages\click\core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\site-packages\dagster_webserver\cli.py", line 212, in dagster_webserver
    with get_possibly_temporary_instance_for_cli(
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\contextlib.py", line 158, in __exit__
    self.gen.throw(typ, value, traceback)
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\site-packages\dagster\_cli\utils.py", line 83, in get_possibly_temporary_instance_for_cli
    with _get_temporary_instance(cli_command, logger) as instance:
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\contextlib.py", line 158, in __exit__
    self.gen.throw(typ, value, traceback)
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\site-packages\dagster\_cli\utils.py", line 32, in _get_temporary_instance
    with tempfile.TemporaryDirectory(dir=os.getcwd()) as tempdir:
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\tempfile.py", line 943, in __exit__
    self.cleanup()
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\tempfile.py", line 947, in cleanup
    self._rmtree(self.name, ignore_errors=self._ignore_cleanup_errors)
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\tempfile.py", line 929, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\shutil.py", line 787, in rmtree
    return _rmtree_unsafe(path, onerror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\shutil.py", line 629, in _rmtree_unsafe
    _rmtree_unsafe(fullname, onerror)
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\shutil.py", line 638, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\tempfile.py", line 919, in onerror
    cls._rmtree(path, ignore_errors=ignore_errors,
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\tempfile.py", line 929, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\shutil.py", line 787, in rmtree
    return _rmtree_unsafe(path, onerror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\shutil.py", line 638, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\tempfile.py", line 919, in onerror
    cls._rmtree(path, ignore_errors=ignore_errors,
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\tempfile.py", line 929, in _rmtree
    _shutil.rmtree(name, onerror=onerror)
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\shutil.py", line 787, in rmtree
    return _rmtree_unsafe(path, onerror)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\shutil.py", line 638, in _rmtree_unsafe
    onerror(os.rmdir, path, sys.exc_info())
  File "C:\ProgramData\Miniconda3\envs\xlwings\Lib\shutil.py", line 636, in _rmtree_unsafe
    os.rmdir(path)
PermissionError: [WinError 32] Prosessen får ikke tilgang til filen fordi den brukes av en annen prosess: 'C:\\Dropbox\\CODE2\\xlwings\\excel-ics\\tmpka29b0dd\\schedules'

What did you expect to happen?

I haven't implemented any dagster elements in the code base, but I wanted to see how the web interface looked like. I didn't expect it to fail immediately. Also, the error message indicates that the error was caused by faulty file handling by dagster_webserver, since the traceback shows a conflict between differentent process trying to access a temporary file created by dagster_webserver.

How to reproduce?

On a Windows 11 PC:
# packages in environment at C:\ProgramData\Miniconda3\envs\xlwings:
#
# Name                    Version                   Build  Channel
alembic                   1.13.1             pyhd8ed1ab_1    conda-forge
aniso8601                 9.0.1              pyhd8ed1ab_0    conda-forge
annotated-types           0.6.0              pyhd8ed1ab_0    conda-forge
anyio                     4.3.0              pyhd8ed1ab_0    conda-forge
arrow                     1.3.0                    pypi_0    pypi
asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
attrs                     23.2.0                   pypi_0    pypi
backoff                   2.2.1              pyhd8ed1ab_0    conda-forge
black                     24.2.0          py311h1ea47a8_0    conda-forge
brotli-python             1.1.0           py311h12c1d0e_1    conda-forge
bzip2                     1.0.8                hcfcfb64_5    conda-forge
c-ares                    1.27.0               hcfcfb64_0    conda-forge
ca-certificates           2024.2.2             h56e8100_0    conda-forge
certifi                   2024.2.2           pyhd8ed1ab_0    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
click                     8.1.7           win_pyh7428d3b_0    conda-forge
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
coloredlogs               14.0               pyhd8ed1ab_3    conda-forge
comm                      0.2.2              pyhd8ed1ab_0    conda-forge
croniter                  2.0.3              pyhd8ed1ab_0    conda-forge
dagit                     1.6.11             pyhd8ed1ab_0    conda-forge
dagster                   1.6.11            pyhd8ed1ab_11    conda-forge
dagster-graphql           1.6.11             pyhd8ed1ab_0    conda-forge
dagster-pipes             1.6.11             pyhd8ed1ab_0    conda-forge
dagster-webserver         1.6.11             pyhd8ed1ab_0    conda-forge
dataclasses               0.8                pyhc8e2a94_3    conda-forge
debugpy                   1.8.1           py311h12c1d0e_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
docstring_parser          0.16               pyhd8ed1ab_0    conda-forge
dotsi                     0.0.3                    pypi_0    pypi
et_xmlfile                1.1.0              pyhd8ed1ab_0    conda-forge
exceptiongroup            1.2.0              pyhd8ed1ab_2    conda-forge
executing                 2.0.1              pyhd8ed1ab_0    conda-forge
fsspec                    2024.3.1           pyhca7485f_0    conda-forge
gql                       3.5.0              pyhd8ed1ab_1    conda-forge
gql-with-requests         3.5.0              pyhd8ed1ab_1    conda-forge
graphene                  3.3                pyhd8ed1ab_0    conda-forge
graphql-core              3.2.3              pyhd8ed1ab_0    conda-forge
graphql-relay             3.2.0              pyhd8ed1ab_0    conda-forge
greenlet                  3.0.3           py311h12c1d0e_0    conda-forge
grpcio                    1.62.1          py311h5bc0dda_0    conda-forge
grpcio-health-checking    1.62.1             pyhd8ed1ab_0    conda-forge
h11                       0.14.0             pyhd8ed1ab_0    conda-forge
httptools                 0.6.1           py311ha68e1ae_0    conda-forge
humanfriendly             10.0            py311h1ea47a8_5    conda-forge
ics                       0.7.2                    pypi_0    pypi
idna                      3.6                pyhd8ed1ab_0    conda-forge
importlib-metadata        7.0.2              pyha770c72_0    conda-forge
importlib_metadata        7.0.2                hd8ed1ab_0    conda-forge
importlib_resources       6.4.0              pyhd8ed1ab_0    conda-forge
intel-openmp              2024.0.0         h57928b3_49841    conda-forge
ipykernel                 6.29.3             pyha63f2e9_0    conda-forge
ipython                   8.22.2             pyh7428d3b_0    conda-forge
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
jinja2                    3.1.3              pyhd8ed1ab_0    conda-forge
jupyter_client            8.6.1              pyhd8ed1ab_0    conda-forge
jupyter_core              5.7.2           py311h1ea47a8_0    conda-forge
libabseil                 20240116.1      cxx17_h63175ca_2    conda-forge
libblas                   3.9.0              21_win64_mkl    conda-forge
libcblas                  3.9.0              21_win64_mkl    conda-forge
libexpat                  2.6.2                h63175ca_0    conda-forge
libffi                    3.4.4                hd77b12b_0  
libgrpc                   1.62.1               h5273850_0    conda-forge
libhwloc                  2.9.3           default_haede6df_1009    conda-forge
libiconv                  1.17                 hcfcfb64_2    conda-forge
liblapack                 3.9.0              21_win64_mkl    conda-forge
libprotobuf               4.25.3               h503648d_0    conda-forge
libre2-11                 2023.09.01           hf8d8778_2    conda-forge
libsodium                 1.0.18               h8d14728_1    conda-forge
libsqlite                 3.45.2               hcfcfb64_0    conda-forge
libxml2                   2.12.5               hc3477c8_0    conda-forge
libzlib                   1.2.13               hcfcfb64_5    conda-forge
mako                      1.3.2              pyhd8ed1ab_0    conda-forge
markdown-it-py            3.0.0              pyhd8ed1ab_0    conda-forge
markupsafe                2.1.5           py311ha68e1ae_0    conda-forge
matplotlib-inline         0.1.6              pyhd8ed1ab_0    conda-forge
mdurl                     0.1.2              pyhd8ed1ab_0    conda-forge
mkl                       2024.0.0         h66d3029_49657    conda-forge
multidict                 6.0.5           py311ha68e1ae_0    conda-forge
mypy                      1.9.0           py311ha68e1ae_0    conda-forge
mypy_extensions           1.0.0              pyha770c72_0    conda-forge
nest-asyncio              1.6.0              pyhd8ed1ab_0    conda-forge
numpy                     1.26.4          py311h0b4df5a_0    conda-forge
openpyxl                  3.1.2           py311ha68e1ae_1    conda-forge
openssl                   3.2.1                hcfcfb64_1    conda-forge
packaging                 24.0               pyhd8ed1ab_0    conda-forge
pandas                    2.2.1           py311hf63dbb6_0    conda-forge
parso                     0.8.3              pyhd8ed1ab_0    conda-forge
pathspec                  0.12.1             pyhd8ed1ab_0    conda-forge
pendulum                  3.0.0           py311h633b200_0    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pip                       24.0               pyhd8ed1ab_0    conda-forge
platformdirs              4.2.0              pyhd8ed1ab_0    conda-forge
prompt-toolkit            3.0.43          py311haa95532_0  
prompt_toolkit            3.0.43               hd3eb1b0_0  
protobuf                  4.25.3          py311hb041b4a_0    conda-forge
psutil                    5.9.8           py311ha68e1ae_0    conda-forge
pthreads-win32            2.9.1                hfa6e2cd_3    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pydantic                  2.6.4              pyhd8ed1ab_0    conda-forge
pydantic-core             2.16.3          py311hc37eb10_0    conda-forge
pygments                  2.17.2             pyhd8ed1ab_0    conda-forge
pyreadline3               3.4.1           py311h1ea47a8_3    conda-forge
pysocks                   1.7.1              pyh0701188_6    conda-forge
python                    3.11.8          h2628c8c_0_cpython    conda-forge
python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
python-dotenv             1.0.1              pyhd8ed1ab_0    conda-forge
python-tzdata             2024.1             pyhd8ed1ab_0    conda-forge
python_abi                3.11                    4_cp311    conda-forge
pytz                      2024.1             pyhd8ed1ab_0    conda-forge
pywin32                   306             py311h12c1d0e_2    conda-forge
pywin32-on-windows        0.1.0              pyh07e9846_2    conda-forge
pyyaml                    6.0.1           py311ha68e1ae_1    conda-forge
pyzmq                     25.1.2          py311h9250fbb_0    conda-forge
re2                       2023.09.01           hd3b24a8_2    conda-forge
requests                  2.31.0             pyhd8ed1ab_0    conda-forge
requests-toolbelt         1.0.0              pyhd8ed1ab_0    conda-forge
rich                      13.7.1             pyhd8ed1ab_0    conda-forge
setuptools                69.2.0             pyhd8ed1ab_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
sniffio                   1.3.1              pyhd8ed1ab_0    conda-forge
sqlalchemy                2.0.28          py311ha68e1ae_0    conda-forge
sqlite                    3.45.2               hcfcfb64_0    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
starlette                 0.37.2             pyhd8ed1ab_0    conda-forge
structlog                 24.1.0             pyhd8ed1ab_0    conda-forge
tabulate                  0.9.0              pyhd8ed1ab_1    conda-forge
tatsu                     5.8.3                    pypi_0    pypi
tbb                       2021.11.0            h91493d7_1    conda-forge
time-machine              2.13.0          py311ha68e1ae_1    conda-forge
tk                        8.6.13               h5226925_1    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
toposort                  1.10               pyhd8ed1ab_0    conda-forge
tornado                   6.4             py311ha68e1ae_0    conda-forge
tqdm                      4.66.2             pyhd8ed1ab_0    conda-forge
traitlets                 5.14.2             pyhd8ed1ab_0    conda-forge
typed-ast                 1.5.5           py311ha68e1ae_1    conda-forge
types-python-dateutil     2.8.19.20240311          pypi_0    pypi
typing-extensions         4.10.0               hd8ed1ab_0    conda-forge
typing_extensions         4.10.0             pyha770c72_0    conda-forge
tzdata                    2024a                h0c530f3_0    conda-forge
ucrt                      10.0.22621.0         h57928b3_0    conda-forge
universal_pathlib         0.2.2              pyhd8ed1ab_0    conda-forge
urllib3                   2.2.1              pyhd8ed1ab_0    conda-forge
uvicorn                   0.29.0          py311h1ea47a8_0    conda-forge
uvicorn-standard          0.29.0               h1ea47a8_0    conda-forge
vc                        14.3                hcf57466_18    conda-forge
vc14_runtime              14.38.33130         h82b7239_18    conda-forge
vs2015_runtime            14.38.33130         hcb4865c_18    conda-forge
watchdog                  4.0.0           py311h1ea47a8_0    conda-forge
watchfiles                0.21.0          py311hc37eb10_0    conda-forge
wcwidth                   0.2.13             pyhd8ed1ab_0    conda-forge
websockets                11.0.3          py311ha68e1ae_1    conda-forge
wheel                     0.42.0             pyhd8ed1ab_0    conda-forge
win_inet_pton             1.1.0              pyhd8ed1ab_6    conda-forge
xlwings                   0.30.15         py311h1ea47a8_0    conda-forge
xz                        5.4.6                h8cc25b3_0  
yaml                      0.2.5                h8ffe710_2    conda-forge
yarl                      1.9.4           py311ha68e1ae_0    conda-forge
zeromq                    4.3.5                h63175ca_1    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               hcfcfb64_5    conda-forge

Deployment type

None

Deployment details

No response

Additional information

No response

Message from the maintainers

Impacted by this issue? Give it a 👍! We factor engagement into prioritization.

[garethbrickman]

The traceback you've provided indicates that there is a PermissionError occurring when the Dagster process is trying to clean up a temporary directory. This error is happening because the process does not have the necessary permissions to delete a file or directory, or because the file or directory is being used by another process.

Here are a few steps you can take to troubleshoot this issue:

1. Check Permissions: Ensure that the user running the Dagster process has the necessary permissions to delete files and directories in the specified location.

2. Check for Open Files: Make sure that no other processes are using the files or directories that Dagster is trying to delete. This can happen if you have a file open in an editor, or if another process is accessing the file.

3. Path in Use: The error message "Prosessen får ikke tilgang til filen fordi den brukes av en annen prosess" translates to "The process cannot access the file because it is being used by another process." This suggests that the directory or file Dagster is trying to remove is locked because it's in use. You can try closing any applications that might be using the files or directories in question.

4. Temporary Directory on Network Drive: If the temporary directory is located on a network drive or a synchronized folder (like Dropbox), there might be synchronization issues that prevent the deletion. Try setting the temporary directory to a local path that is not being synchronized or monitored by other services.

5. Restart the System: In some cases, simply restarting the system can free up the locked files or directories and resolve the issue.

[TobiSan5]

Yes, the problem could very well be caused by the file in question being on a syncronized folder. My project folder is within my Dropbox folder. Dropbox came up with a notification about a file being removed from Dropbox. I later followed the tutorial on docs.dagster.io, and ran the web UI with the command dagster dev, and no such error occured.

fre. 22. mars 2024 kl. 14:31 skrev Gareth Brickman @.***

:

The traceback you've provided indicates that there is a PermissionError occurring when the Dagster process is trying to clean up a temporary directory. This error is happening because the process does not have the necessary permissions to delete a file or directory, or because the file or directory is being used by another process.

Here are a few steps you can take to troubleshoot this issue:

1.

Check Permissions: Ensure that the user running the Dagster process has the necessary permissions to delete files and directories in the specified location. 2.

Check for Open Files: Make sure that no other processes are using the files or directories that Dagster is trying to delete. This can happen if you have a file open in an editor, or if another process is accessing the file. 3.

Path in Use: The error message "Prosessen får ikke tilgang til filen fordi den brukes av en annen prosess" translates to "The process cannot access the file because it is being used by another process." This suggests that the directory or file Dagster is trying to remove is locked because it's in use. You can try closing any applications that might be using the files or directories in question. 4.

Temporary Directory on Network Drive: If the temporary directory is located on a network drive or a synchronized folder (like Dropbox), there might be synchronization issues that prevent the deletion. Try setting the temporary directory to a local path that is not being synchronized or monitored by other services. 5.

Restart the System: In some cases, simply restarting the system can free up the locked files or directories and resolve the issue.

— Reply to this email directly, view it on GitHub https://github.com/dagster-io/dagster/issues/20654#issuecomment-2015106324, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABY7TYSEMJYLN7UK3J6K55DYZQXEXAVCNFSM6AAAAABFC5NB7WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMJVGEYDMMZSGQ . You are receiving this because you authored the thread.Message ID: @.***>

-- Med hilsen / With regards from,

Torbjørn Wikestad +47 41 26 41 07