MSBuild should handle paths longer than MAX_PATH

[ariccio]

As seen in Fix 260 character file name length limitation, there's quite a bit of support for better _longer-than-MAX_PATH-filename_ handling.

Will Buik's response w/regard to fixing it was:

We understand that this can be a frustrating issue, however, fixing it requires a large and complicated architectural change across different products and features including Visual Studio, TFS, MSBuild and the .NET Framework. Dedicating resources to this work item would come at the expense of many other features and innovation.

I wondered, how big can those architectural changes be (for MSBuild)?

According to Naming Files, Paths, and Namespaces: Maximum Path Length Limitation, (which I have mirrored here ):

The Windows API has many functions that also have Unicode versions to permit an extended-length path for a maximum total path length of 32,767 characters. This type of path is composed of components separated by backslashes, each up to the value returned in the lpMaximumComponentLength parameter of the GetVolumeInformation function (this value is commonly 255 characters). To specify an extended-length path, use the "\\?\" prefix. For example, "\\?\D:\very long path". [other stuff]

Because you cannot use the "\\?\" prefix with a relative path, relative paths are always limited to a total of MAX_PATH characters.

(emphasis mine)

...Which means that wherever MSBuild uses full paths, we can should be able to just prepend "\\?\" to ask for a long path.

Of course, we'd need to rework the existing path-length-workaround hack.

This won't fix the whole ecosystem, but it'll get us just one step closer.

I'd like to fix this myself (it seems simple enough), so in accordance with:

• Contributions must be discussed with the team first, or they will likely be declined. As our process matures and our experience grows, the team expects to take larger contributions.
• Only contributions referencing an approved Issue will be accepted.

...this is the suggested issue.

I'm a native developer at heart, not an experienced C# developer, but this shouldn't require anything crazy.

Direct references to MAX_PATH:

• NativeMethods.cs#L15
• NativeMethodsShared_Tests.cs#L53
• NativeMethodsShared.cs#L318
• NativeMethodsShared.cs#L540
• NativeMethodsShared.cs#L750
• FileState.cs#L293
• ComReference.cs#L460
• TargetsFile_Test.cs#L2041

[dasMulli]

16.0 / VS 2019 seems like a good opportunity to get this in.. given that msbuild (and VS?) now targets 4.7.2 instead of 4.6. At least having https://github.com/Microsoft/msbuild/pull/3503 would be great.

[tsibiski]

[robinabganpat]

I'm currently using Git with a few nested submodules for an enterprise project. Trying to build a project in the deepest nested submodule (depth 2) is already throwing the path length error. When issues like these limit the separation of responsibilities and the structure of your project overall, really sucks.

[mvsrinivasan]

This bug will be fixed in year 201. Oops I assigned only 3 characters for handling years!

[elielia]

Who developed this stuff? A high school computer science graduate? What is the easiest way to hack it? I cannot believe my build is stuck because of a long path. Unbelievable

[mvsrinivasan]

@elielia : Try creating symlinks with shorter path pointing to your actual long paths.

[am11]

@elielia, one approach is to use good old subst command to create a virtual drive from project directory and reset the nesting. Run MSBuild in the virtual drive and delete the drive afterwards:

#!/usr/bin/env pwsh

subst W: c:\work\git\myLib

Push-Location W:\
msbuild /p:Configuration=Release myLib.sln
Pop-Location

subst W: /d

[skiryazov]

These workarounds are the same as saying "well, just checkout your repo directly into your C: drive"; they don't help when your repository's inner structure exceeds the limit.

Yes, you could also refactor the repo so that it stays within the limit but - guess what - we've already thought of that and it's far from ideal.

[XVilka]

Another solution - just use the proper build system, something like CMake or similar. Coupled with Ninja and clang-cl (supposing you are using C/C++) it will greatly surprise you, like it did Google Chrome and Mozilla Firefox developers, addionally provide you helpful tools like ASAN, UBSAN, etc.

[michaelheilmann]

Another solution - just use the proper build system, something like CMake or similar. Coupled with Ninja and clang-cl (supposing you are using C/C++) it will greatly surprise you, like it did Google Chrome and Mozilla Firefox developers, addionally provide you helpful tools like ASAN, UBSAN, etc.

Well, the first time I encountered this issue was exactly when migrating to CMake. CMake certainly is a great asset, and CMake makes working around the problem more tractable than other tools, but the problem is not eliminated simply by using CMake. Just saying, for that people do not see CMake as the a golden bullet, it still generates project files for msbuild after all.

[XVilka]

Modern CMake can generate ninja files instead.

On Fri, Jan 25, 2019, 7:54 AM Michael Heilmann <notifications@github.com wrote:

Another solution - just use the proper build system, something like CMake or similar. Coupled with Ninja and clang-cl (supposing you are using C/C++) it will greatly surprise http://blog.llvm.org/2018/11/30-faster-windows-builds-with-clang-cl_14.html you, like it did Google Chrome http://blog.llvm.org/2018/03/clang-is-now-used-to-build-chrome-for.html and Mozilla Firefox https://groups.google.com/forum/m/#!topic/mozilla.dev.platform/wwO48xXFx0A developers, addionally provide you helpful tools like ASAN, UBSAN, etc.

Well, the first time I encountered this issue was exactly when migrating to CMake. CMake certainly is a great asset, and CMake makes working around the problem more tractable than other tools, but the problem is not eliminated simply by using CMake. Just saying, for that people do not see CMake as the a golden bullet, it still generates project files for msbuild after all.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/Microsoft/msbuild/issues/53#issuecomment-457403021, or mute the thread https://github.com/notifications/unsubscribe-auth/AAMZ_SK2pEH67yk4b0NMbt68kUZqk8ITks5vGkefgaJpZM4D3alK .

[mdealer]

What I was referring to was pretty specific to MSBuild and not necessarily a root cause for the larger ecosystem. And I get the frustration, we are affected by this here as well. But there are a lot of barriers for this particular problem for it to be fully solved. For example: https://github.com/search?q=MAX_PATH&type=Code

That search alone has over 1M code hits. Through a modern lens I don't think you'll get an argument from anyone that this was a good idea, but we are trying to address it. Our hope is to have all of these solved for MSBuild in our codebase for the next major release. Unfortunately, even that may not fully help you if one of your dependencies still has path limits.

Put it waaaay up in the prio list and start working. 1M code hits... Jesus... It'll be totally fine if it won't work at first. We'll let you know how it goes in any case.

[rainersigwald]

MAX_PATH in MSBuild 16.0

tl;dr: Install Visual Studio 2019 Preview 2, enable long paths, and things are slightly better.

What's changed

As of Visual Studio 2019 Preview 2 (which includes MSBuild 16.0.360-preview), MSBuild.exe now (as of https://github.com/Microsoft/msbuild/pull/3507) opts into support for long paths.

This is dependent on .NET Framework, operating system, and host-executable support, so there is a big prerequisite: you must enable long paths in Windows by using Group Policy or setting a registry key. There's Windows documentation about the registry key.

Does that mean things work?

No, for all the reasons detailed in comments above: just because MSBuild works, doesn't mean your build will, because many other tools are involved.

devenv.exe, the main Visual Studio process, does not yet opt into support. That means only command-line builds will be affected by the MSBuild changes.

Where should I follow up?

If you encounter a long-path problem with a specific executable, tool, or task, ideally report it to the owner of that tool. For instance, I filed https://github.com/dotnet/roslyn/issues/32804 because the C# and VB compilers need to make similar changes to what MSBuild did.

If the task is shipped from this repo, please file a new issue in this repo.

The overall "Visual Studio should support long paths" item is https://developercommunity.visualstudio.com/idea/351628/allow-building-running-and-debugging-a-net-applica.html.

[rainersigwald]

I'm going to close this as "done as much as we can do for 16.0". See the comment above for the many caveats involved, and what to do if you find a specific MAX_PATH problem.

[terrajobst]

Agreed. We're clearly interested in addressing the issue, but we can't drive issues that are stated that broadly.

[dsplaisted]

@rainersigwald @KathleenDollard @mairaw Can we include the info from Rainer's comment in the Dev16 release notes and documentation?

[tcrosen]

tl;dr: Install Visual Studio 2019 Preview 2, enable long paths, and things are slightly better

Is there a standalone MSBuild Tools equivalent to this? I've been able to avoid installing VS on my 10+ build agents so far and I don't plan on starting now with a preview release just to fix this silly problem.

[aolszowka]

@rainersigwald was this expected to be fixed in both the 32bit and 64bit (amd64\msbuild.exe) msbuild binaries? I am getting interesting behavior wherein the 64 bit version throws this error:

"S:\Builds\GenerateInterop-60R39\WorkingDirectory\src\Dotnet\Source\Framework\Interop\Project\ComputersUnlimited.Interop.sln" (Restore target) (1) ->
(Restore target) ->
  C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\CommonExtensions\Microsoft\NuGet\NuGet.targets(121,5): error : The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters. [S:\Builds\GenerateInterop-60R39\WorkingDirectory\src\Dotnet\Source\Framework\Interop\Project\ComputersUnlimited.Interop.sln]

Whereas using the 32bit version has no such qualms (restores and builds without issue).

Here's the version from the 64bit and 32bit versions (same):

Microsoft (R) Build Engine version 16.1.76+g14b0a930a7 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

I can open a new issue if need be but wanted to start here. If this is something I need to get off to the NuGet guys I can do that as well.

[mahaase]

Has anyone ever seen/verified, that this fix still works?

I enabled long paths in gpedit. I checked msbuild.exe resource stuff, still fine. I use MSBuild v16.2, but the long path issues still exists.

"...maximale Pfadlimit des Betriebssystems. Der vollqualifizierte Dateiname muss weniger als 260 Zeichen umfassen"

The issue occurs inside a build-system, which handles the paths (directory-names a.s.o.) by itself, no way/easy way to change them.

[inf9144]

@mahaase same problem here. 32bit is still working, 64bit does show the max_path error. Tested with newest visual studio build tools on newest windows 10. Thanks for the tip with 32bit. I was nearly giving up and taking myself to a knit.

[inf9144]

@rainersigwald could you please reopen this ticket because of the bug in 64bit?

[KirillOsenkov]

I think this should be a new issue, with specifics about 64-bit not working as expected

[mahaase]

I think this should be a new issue, with specifics about 64-bit not working as expected

I opened a new ticket for 64-bit variant. I hope the issue will solved soon, now. This bug is a show-stopper.

[Piedone]

Worth linking to it: https://github.com/microsoft/msbuild/issues/5331

[brignolij]

Hi, I'm facing the issue with path limit using AWS codebuild (meaning buildspec.yml file) , msbuild and with docker image (mcr.microsoft.com/dotnet/framework/sdk, https://hub.docker.com/_/microsoft-dotnet-framework-sdk). Any suggestion to make it work using only powershell/cmd command ? more documentation : https://aws.amazon.com/fr/blogs/devops/creating-ci-cd-pipelines-for-asp-net-4-x-with-aws-codepipeline-and-aws-elastic-beanstalk/

[rainersigwald]

@brignolij I'm sorry, I don't know whether the container hosts need the long-path configuration, or just the container images. If setting the registry key inside the container doesn't work, I think you'd have to ask AWS support.

[brignolij]

@rainersigwald Thanks for your answer. I already have a issue open with AWS support, they suggest me to check on msbuild support. I have a project with long path. (lot of files ,folders). So i really want to make it work with long path. i test this following command into my buildsepc Set-ItemProperty 'HKLM:\System\CurrentControlSet\Control\FileSystem' -Name 'LongPathsEnabled' -value 1 but this command requiere a system restart to be effective and this not possible on build system like awscodebuild. I'm open to any suggestion. Also is application manifest file required for .net framework mvc app? is yes where should we add ?

<application xmlns="urn:schemas-microsoft-com:asm.v3">
    <windowsSettings xmlns:ws2="http://schemas.microsoft.com/SMI/2016/WindowsSettings">
        <ws2:longPathAware>true</ws2:longPathAware>
    </windowsSettings>
</application>

Regards

[PhDittmann]

@rainersigwald was this expected to be fixed in both the 32bit and 64bit (amd64\msbuild.exe) msbuild binaries? I am getting interesting behavior wherein the 64 bit version throws this error:

"S:\Builds\GenerateInterop-60R39\WorkingDirectory\src\Dotnet\Source\Framework\Interop\Project\ComputersUnlimited.Interop.sln" (Restore target) (1) ->
(Restore target) ->
  C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\CommonExtensions\Microsoft\NuGet\NuGet.targets(121,5): error : The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters. [S:\Builds\GenerateInterop-60R39\WorkingDirectory\src\Dotnet\Source\Framework\Interop\Project\ComputersUnlimited.Interop.sln]

Whereas using the 32bit version has no such qualms (restores and builds without issue).

Here's the version from the 64bit and 32bit versions (same):

Microsoft (R) Build Engine version 16.1.76+g14b0a930a7 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

I can open a new issue if need be but wanted to start here. If this is something I need to get off to the NuGet guys I can do that as well.

I just had the same problem.

C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\Bin\amd64\MSBuild.exe

Microsoft (R) Build Engine version 16.9.0+57a23d249 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

fails with 'path too long' while

C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\Bin\MSBuild.exe

Microsoft (R) Build Engine version 16.9.0+57a23d249 for .NET Framework
Copyright (C) Microsoft Corporation. All rights reserved.

(without \amd64) works perfectly for the same .sln.

[matteius]

Ran into this issue with a report to the pipenv backlog that I triaged: pip install fastchunking on windows fails outright with this: https://github.com/pypa/pip/issues/11231

https://docs.microsoft.com/en-us/cpp/error-messages/tool-errors/linker-tools-error-lnk1104?view=msvc-170#paths-that-are-too-long

I have enabled the registry key for Windows 10 long paths, but the 2019 Visual Studio linker don't care about that. Is there any path of resolution other than move the temp directory to a shorter path so that hopefully the compiled file path lengths don't exceed 260?

[hsdk123]

Same problem here - unsure why this was closed.

[rainersigwald]

@hsdk123 please file a new bug with details of the problem you're seeing.

[walbourn]

The current versions of MSBuild long-path aware in the manifest:

  <application xmlns="urn:schemas-microsoft-com:asm.v3">
      <windowsSettings xmlns:ws2="http://schemas.microsoft.com/SMI/2016/WindowsSettings">
          <ws2:longPathAware>true</ws2:longPathAware>
      </windowsSettings>
  </application>

I'm not sure WHEN this was added...

[rainersigwald]

@walbourn https://github.com/dotnet/msbuild/issues/53#issuecomment-459062618.

I'm going to lock this thread, since the core issues are long fixed and anything that crops up now should get a new issue.