frerich / clcache

A compiler cache for MSVC, much like ccache for gcc
Other
326 stars 83 forks source link

PermissionError: [WinError 5] Access is denied if using build multithreads > 3 #342

Open shijunjing opened 5 years ago

shijunjing commented 5 years ago

Hello, I'm enabling the clcache in the Uefi firmware MSVC build toolchain. I've enable thed ccache in our GCC/CLang toolchains and I hope the clcache can work in our MSVC toolchain as well. I built the clcache through the pyinstaller and I directly hard code the clcache_main.exe as my firmware VS2015 build CL and build in VS2015 cmd environment. My firmware build is driven by Uefi python build script. In my testing, I found if I build with single thread or threads <= 3, the clcache can work well. However, if I build with threads > 4, the clcache will raise the PermissionError: [WinError 5] Access is denied, below is an example. The more threads I use, the easier this issue raise.

Is there any missing configure in my build steps? Any suggestion is appreciated.

[11116] Failed to execute script clcache_main Traceback (most recent call last): File "clcache_main.py", line 2, in File "site-packages\clcache__main.py", line 1589, in main File "site-packages\clcache__main.py", line 1616, in processCompileRequest File "site-packages\clcache\main__.py", line 1668, in scheduleJobs File "concurrent\futures_base.py", line 425, in result File "concurrent\futures_base.py", line 384, in get_result File "concurrent\futures\thread.py", line 57, in run File "site-packages\clcache__main.py", line 1689, in processSingleSource File "site-packages\clcache__main.py", line 1719, in processDirect File "site-packages\clcache__main.py", line 1507, in processCacheHit File "site-packages\clcache\main.py", line 749, in exit File "site-packages\clcache\main.py", line 659, in save File "contextlib.py", line 119, in exit File "site-packages\atomicwrites\init__.py", line 159, in _open File "site-packages\atomicwrites\init.py", line 190, in commit File "site-packages\atomicwrites\init__.py", line 93, in replace_atomic File "site-packages\atomicwrites\init.py", line 75, in _replace_atomic File "site-packages\atomicwrites\init__.py", line 70, in _handle_errors PermissionError: [WinError 5] Access is denied. NMAKE : fatal error U1077: 'C:\steven\clcache-fork\pyinstaller\dist\clcache_main\clcache_main.exe' : return code '0xffffffff' Stop.

izmmisha commented 5 years ago

Hello, take a look to https://github.com/frerich/clcache/issues/339, same problem here.

shijunjing commented 5 years ago

Moving clcache dir out of user profile dir doesn't work for me. I moved the C:\steven\clcache-fork\pyinstaller\dist\clcache_main\ to C:\clcache_main\, and the build still fail with threads > 4. Below is the failure log:

[7000] Failed to execute script clcache_main Traceback (most recent call last): File "clcache_main.py", line 2, in File "site-packages\clcache__main.py", line 1589, in main File "site-packages\clcache__main.py", line 1616, in processCompileRequest File "site-packages\clcache\main__.py", line 1668, in scheduleJobs File "concurrent\futures_base.py", line 425, in result File "concurrent\futures_base.py", line 384, in get_result File "concurrent\futures\thread.py", line 57, in run File "site-packages\clcache__main.py", line 1689, in processSingleSource File "site-packages\clcache__main.py", line 1719, in processDirect File "site-packages\clcache__main.py", line 1507, in processCacheHit File "site-packages\clcache\main.py", line 749, in exit File "site-packages\clcache\main.py", line 659, in save File "contextlib.py", line 119, in exit File "site-packages\atomicwrites\init__.py", line 159, in _open File "site-packages\atomicwrites\init.py", line 190, in commit File "site-packages\atomicwrites\init__.py", line 93, in replace_atomic File "site-packages\atomicwrites\init.py", line 75, in _replace_atomic File "site-packages\atomicwrites\init__.py", line 70, in _handle_errors PermissionError: [WinError 5] Access is denied. NMAKE : fatal error U1077: 'C:\clcache_main\clcache_main.exe' : return code '0xffffffff' Stop.

izmmisha commented 5 years ago

you should move clcache dir, not clcache binary. It can be configured through CLCACHE_DIR environment

shijunjing commented 5 years ago

Thank you! It works with threads > 4 after I set set CLCACHE_DIR=C:\clcache_dir.

Is this issue a known limitation by design? Do we have plan to fix it?

frerich commented 5 years ago

I think this might warrant an entry in the 'Known Issues' section of the Wiki.

benjaminfuchs commented 5 years ago

@frerich I added an idea to fix the issue in this comment https://github.com/frerich/clcache/pull/334#issuecomment-452475038

boris-papevis commented 5 years ago

I can confirm that the retry workaround from @shijunjing is working great. @frerich, please consider merging this one.