search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
15.2k stars 4.72k forks source link

Source generators generate longer paths than anybody can handle #106702

Open ygoe opened 2 months ago

ygoe commented 2 months ago

Description

Source generators are a nice thing in .NET 8, but the files they create live in subdirectories of subdirectories of more subdirectories, each one with a longer name than the other. You can imagine where it leads: paths longer than 260 characters. I've hit this issue with decently short namespace lengths already. Then it's impossible to inspect the generated files with Visual Studio or Notepad++. Only VS Code can still open them somehow, but not show their full name anymore.

Reproduction Steps

Use a source generator on a project with namespaces longer than 50 characters in a project with a base path of at least 40 characters.

Expected behavior

The generated files should be in shorter directories.

Actual behavior

The generated files are in directories named after the namespace and type of the generator + the namespace and type of the processed type + the "obj\Debug\net8.0\generated" prefix.

Regression?

Probably not.

Known Workarounds

Source generators can try to delay the issue by creating shorter file names, starting with a short hash of the namespace. But the directories are still long.

Configuration

No response

Other information

https://developercommunity.visualstudio.com/t/Allow-file-names-longer-than-259-charact/10711192

julealgon commented 2 months ago

@ygoe could you provide a concrete example including the file path it generates, and your proposed shortened version?

ygoe commented 2 months ago

This is the original name I would have gotten:

C:\Source\DevTools\Unclassified.ObjectData\Unclassified.ObjectData.UnitTests\obj\Debug\net8.0\generated\Unclassified.ObjectData.Generators\Unclassified.ObjectData.Generators.JsonSerializationGenerator\Unclassified.ObjectData.UnitTests.EmailAddressWithName.Json.g.cs

*) Possible for me to change, but makes the path structure a lot worse **) Not under my control with reasonable measures or at all

It has 265 characters. Not exceeding the limit by much, but too long is too long, no matter how far. The file can't be opened in many applications. Stupid path length limit, a thing of the ancient past actually. But nobody cares, so it's still there.

This is my workaround with a custom truncated hash of the namespace:

C:\Source\DevTools\Unclassified.ObjectData\Unclassified.ObjectData.UnitTests\obj\Debug\net8.0\generated\Unclassified.ObjectData.Generators\Unclassified.ObjectData.Generators.JsonSerializationGenerator\kKSf.EmailAddressWithName.Json.g.cs

Length: - 33 + 4 characters. But should somebody decide to use my generator in a project that has a longer base path, it can still get ugly when the generated files need to be inspected.

I'm not sure how to simplify this, but the two levels I cannot influence are the first to consider: (bold text is hard to see here)

C:\Source\DevTools\Unclassified.ObjectData\Unclassified.ObjectData.UnitTests\obj\Debug\net8.0\generated\Unclassified.ObjectData.Generators\Unclassified.ObjectData.Generators.JsonSerializationGenerator\Unclassified.ObjectData.UnitTests.EmailAddressWithName.Json.g.cs

Maybe the second one of them could simply be left out, that would save a lot of characters. As the author of the generator project, I can make sure not to generate conflicting file names myself.

Or both could be shortened to their type name without the namespace and counters appended if names exist multiple times.

Clockwork-Muse commented 2 months ago

generated\Unclassified.ObjectData.Generators\Unclassified.ObjectData.Generators.JsonSerializationGenerator\Unclassified.ObjectData.UnitTests.EmailAddressWithName.Json.g.cs

... I don't know if this is part of the standard, but maybe the path here should just be an 8-character hash or something? Although it would need tool support to be able to switch to the generated file intelligibly....

C:\Source\DevTools: Where my developer tools projects are located

Side note: It's recommended to keep repositories in your user profile when possible (keeping it out of a world-writable/root directories for security purposes if nothing else).

ygoe commented 2 months ago

Side note:

I know Microsoft's recommendations on paths and intentionally discard them. I like to keep the entirety of my digital data in directories that are not related to the operating system, for reasons of safety, portability and recoverability. The larger parts are still on the D:\ HDD anyway until I get myself an even bigger SSD. As the only user of my computers this is totally fine. Anyway, following that recommendation in this case would make it even harder to avoid exceeding the maximum path length, adding 11 more characters. Think of the old days of "C:\Dokumente und Einstellungen"… I've made my share of experiences with that.

ygoe commented 2 months ago

I don't know if this is part of the standard

I've edited my comment above to indicate which parts of the path are not under my control. The two levels I mentioned as possible things to change are fully not under my control. So I need you to change it.

julealgon commented 2 months ago

Thanks for sharing that example @ygoe , it does put the problem into perspective and I tend to agree with you.

Just wanted to throw some things here that might help, some of which I've used myself in the past.

  1. You can map your root folder to a special drive letter in Windows using subst.

In your case, it would look like:

subst X: C:\Source\DevTools

This is obviously a workaround solution, but it has helped me in the past as it does save quite a few characters from the 260 max.

  1. Regarding obj\Debug\net8.0\generated, I believe you should be able to customize the Debug/Release directory names with some MSBuild property. I know the target framework (and runtime identifier) can be hidden using 
<PropertyGroup>
  <AppendTargetFrameworkToOutputPath>false</AppendTargetFrameworkToOutputPath>
  <AppendRuntimeIdentifierToOutputPath>false</AppendRuntimeIdentifierToOutputPath>
</PropertyGroup>

https://learn.microsoft.com/en-us/visualstudio/ide/how-to-change-the-build-output-directory?view=vs-2022&tabs=csharp#change-the-build-output-directory-2

This might be useful if you are not multi targeting to save a few more characters.

This max path problem is a very silly issue that has plagued the Windows ecosystem for years though. It's sad to me that tools are still relying on this legacy max value, including even Windows tools (and Windows itself in cases).

Clockwork-Muse commented 2 months ago

This is obviously a workaround solution, but it has helped me in the past as it does save quite a few characters from the 260 max.

If you're going to do this, might as well set up WSL and devcontainers, and do everything in Linux, where all of this becomes a non-issue.

Think of the old days of "C:\Dokumente und Einstellungen"

You don't want to stick it in the modern documents folder anyways, which is synced to OneDrive (by default), and which wouldn't take well to build/debug cycles. Stick it in something like %UserProfile%\repositories.

I like to keep the entirety of my digital data in directories that are not related to the operating system, for reasons of safety, portability and recoverability.

... a big part of the operating system is ensuring exactly those guarantees, and moving them outside user directories actually works against that (too, your best option for recoverability for code is going to be persisting it to a cloud repo). For everything else, a backup drive is a good idea.

ygoe commented 2 months ago

Thanks for the hints about the project properties, I might well disable the framework part as I'm not multi-targeting here. This could also make packaging scripts simpler as they don't need to consider the current framework version anymore.

... a big part of the operating system is ensuring exactly those guarantees ...

Safety: You never know when you get spaces in such system directories, and they tend to mess up build scripts badly. Portability: I work with multiple computers and some of them have different user names. Having a constant source folder removes a class of annoyances and diff noise when syncing files. Recoverability: When Windows goes away for unexpected reasons, I want to access the files on my disks without first hacking all ACLs to let me in. Had that a few times. And if Windows comes back, the ACLs are left open. There's no use in that.

This is getting a bit off-topic, but I wanted to explain my decision. And it seems I'm not alone, I've seen specifically that C:\Source folder quite a few times already. Nice side effect: There's no personal data leaking out through debug symbols in your builds. Because these contain absolute paths from the build environment.

dotnet-policy-service[bot] commented 2 months ago

Tagging subscribers to this area: @dotnet/area-system-runtime See info in area-owners.md if you want to be subscribed.

tannergooding commented 2 months ago

I'd recommend enabling long paths: https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry#enable-long-paths-in-windows-10-version-1607-and-later. Roslyn and many other parts of .NET got support for long paths back in 2019: https://github.com/dotnet/roslyn/issues/32804, NuGet since 4.8 https://learn.microsoft.com/en-us/nuget/reference/cli-reference/cli-ref-long-path. The VS project system still requires the directory name be less than 248 and the directory + file name to be less than 260, but there's a tracking issue (which you linked in the top post) for that and it shouldn't impact the bin/obj outputs which aren't normally part of the solution. The folder view works much better with long paths, but still has a couple limitations, I'd recommend upvoting the relevant issues on VS Feedback for that support to be added.

Source generators only generate files if /p:EmitCompilerGeneratedFile=true. You can then customize the output location using /p:CompilerGeneratedFilesOutputPath=... which defaults to $(IntermediateOutputPath)/generated (https://learn.microsoft.com/en-us/dotnet/csharp/roslyn-sdk/source-generators-overview), just as you can custom other aspects of the binary, object, publish and other artifact output locations (https://learn.microsoft.com/en-us/dotnet/core/sdk/artifacts-output, https://learn.microsoft.com/en-us/dotnet/core/project-sdk/msbuild-props, https://learn.microsoft.com/en-us/visualstudio/msbuild/common-msbuild-project-properties?view=vs-2022, etc)

There are many ways in which you can shorten or otherwise customize the path if that's necessary for your particular setup. The default, however, needs to be that unique and understandable file names are generated.

dotnet-policy-service[bot] commented 2 months ago

Tagging subscribers to this area: @dotnet/area-meta See info in area-owners.md if you want to be subscribed.