dotnet / roslyn

The Roslyn .NET compiler provides C# and Visual Basic languages with rich code analysis APIs.
https://docs.microsoft.com/dotnet/csharp/roslyn-sdk/
MIT License
18.77k stars 3.99k forks source link

Filesystem pollution in `/tmp/` #73795

Closed lonix1 closed 1 month ago

lonix1 commented 2 months ago

I didn't know which repo to use; I hope this is the right one.

I'm using VSCode on linux, where the /tmp/ directory is for ephemeral stuff; I assume it's the same for mac. Nonetheless, it is customary for programmes to place files in there in an orderly manner.

.NET however places dozens, sometimes hundreds, of directories in there:

e60c456d-f767-4de4-aa80-95caa3e19e5b/
c31c0e23-3700-4692-a0c6-5ff56ea063d8/
cbfffc3d-dd0f-4059-ae96-e882719684fd/
72fa8d52-0773-4be2-9145-bba8604b0ec8/
c2fd93e3-d5f0-48e0-8e56-4e9c74d1f910/
97337a02-5c04-4712-a20a-8bcd718ee3c6/
1449b70c-c6ad-4135-a707-abbc0d461f47/
1e4fdd31-d714-49be-a24b-30b8f84ddeda/
a938dcab-d8a0-425d-b6b3-07f6a7571909/
0fca510e-26b2-46b6-aec1-7b5734d77ce7/
4e57ea5d-ba66-4750-b3de-4f33c9ee69d9/
bd05154b-137e-4b57-89be-7f0e2c45e17d/
93124a82-6861-40e4-b7d7-223d6dc12960/
685d7dde-02a1-443b-8c3a-8ae257fa86f9/
0f410e1f-169a-4f9c-9c3b-ffec86eb280e/
df66c68d-1049-429f-a25e-9db92493589f/
7c7ca279-71d5-4b4c-9b33-cfd9a341fccf/
dddb7114-6331-4443-8fe8-ed7d00a05d98/
a67bb117-97e4-49b6-ba8e-3d2e2e0d314d/
391ad502-3439-4530-992f-6b07c28cb538/
.dotnet/
MSBuildTempa/
NuGetScratcha/
VBCSCompiler/
VS/
vscode-typescript1000/
clr-debug-pipe-b5187e-8d813e1c-in|
clr-debug-pipe-f5a892-f6c70223-out|
clr-debug-pipe-5ddf66-50801f14-in|
clr-debug-pipe-5ee318-a3ebf041-out|
clr-debug-pipe-fa5c84-26a5212d-in|
clr-debug-pipe-6bdf48-7a1a6af5-out|
clr-debug-pipe-f8adc1-0a614cb4-in|
clr-debug-pipe-c9a869-afcad883-out|
clr-debug-pipe-6c7e1a-08eb9d9e-in|
clr-debug-pipe-8fcfba-74fb0027-out|
clr-debug-pipe-dd4d4a-ca3a5fed-in|
clr-debug-pipe-9fe4b3-05c3b713-out|
clr-debug-pipe-52ec9b-f5b59528-in|
clr-debug-pipe-f9730f-4fd7963f-out|
clr-debug-pipe-84cdee-8d3b8002-in|
clr-debug-pipe-19d0d6-927d20d9-out|
clr-debug-pipe-506e87-98e374b3-in|
clr-debug-pipe-30f948-0c5d51c9-out|
clr-debug-pipe-bc3758-1612693a-in|
clr-debug-pipe-b4fe4a-0ad80716-out|
dotnet-diagnostic-f9d9a2-e3e914e3-socket=
dotnet-diagnostic-b6b2dc-a28ff87b-socket=
dotnet-diagnostic-cbb24f-85ec6873-socket=
dotnet-diagnostic-66ac4b-165dda49-socket=
dotnet-diagnostic-8ec12c-31466b49-socket=
dotnet-diagnostic-595545-ee6711b2-socket=
dotnet-diagnostic-319ae6-602a5576-socket=
dotnet-diagnostic-47a525-9e8f2d27-socket=
dotnet-diagnostic-7f15aa-a6958128-socket=
dotnet-diagnostic-78a4c8-1ca2f6c5-socket=
dotnet-diagnostic-1402b7-60e86a1d-socket=
dotnet-diagnostic-a51cc0-0ca962bc-socket=
dotnet-diagnostic-fda3e8-aa45a385-socket=
dotnet-diagnostic-7e3824-76e16196-socket=
dotnet-diagnostic-dc188b-4ff0bd42-socket=
dotnet-diagnostic-6babeb-d4d8be40-socket=
dotnet-diagnostic-7af5a7-4f55732c-socket=
dotnet-diagnostic-b7d7ab-edc5ff20-socket=
dotnet-diagnostic-9ac7f0-6d838f85-socket=
dotnet-diagnostic-01a0cf-20c22909-socket=
MSBuild08cc5=
MSBuildb688c=
MSBuildb1cde=
MSBuild194b5=
MSBuild8a233=
MSBuild56c7b=
MSBuild2331f=
MSBuildae7b2=
MSBuilde916f=
MSBuild05ea0=
MSBuild4019a=
MSBuilde5608=
MSBuildfa0bf=
MSBuild3f1bc=
MSBuild4aa4d=
MSBuild81e97=
MSBuildeb869=
MSBuild3f702=
MSBuilde0700=
MSBuild422fe=

And that's just a small sample. Of each category there are maybe hundreds of entries.

It's sort of "filesystem pollution", and makes it hard to use /tmp/. I've become used to it over the years, but we should probably find a better way. It affects our interaction with all other programmes which use /tmp/ as well as our own stuff because we also use /tmp/.

It would be better to do as other programmes do: create a /tmp/microsoft/ directory and place everything in there.

(If this is due to multiple .NET tools, please let me know which, so I can copy this issue to those repos too? Thanks.)

CyrusNajmabadi commented 2 months ago

VBCSCompiler is likely roslyn.

Could you give some examples of what are in;

  1. The directories with Guid names (e.g.e60c456d-f767-4de4-aa80-95caa3e19e5b/)
  2. VS/
  3. clr-debug-pipe-b5187e-8d813e1c-in
  4. dotnet-diagnostic-f9d9a2-e3e914e3-socket=

It will help track down the relevant teams (if not us).

lonix1 commented 2 months ago

Murphy's Law I check some of those directories and they're empty :smile: ... I should probably use various tools for a few hours to see changes.

These seem to be zero-byte files:

$ tree /tmp/VS/

VS
└── AnalyzerAssemblyLoader
    └── 062743844ff84b278c6a1b7b371573ab
        ├── 032157bc-89b1-4e73-93e3-18c0fba3f40b
        │   └── xunit.analyzers.dll
        ├── 060ba7bc-3f62-47a5-8f4b-3a642b5b6334
        │   └── Microsoft.CodeAnalysis.CodeStyle.Fixes.dll
        ├── 3a04c200-391f-4a6a-9d11-54c89763e567
        │   └── Microsoft.CodeAnalysis.CodeStyle.dll
        ├── 3bf23cdb-9df7-422d-9b81-42ba61eb537b
        │   └── Microsoft.CodeAnalysis.CSharp.CodeStyle.dll
        ├── 3c53bb73-5375-4464-aa38-c4852b5e8887
        │   └── StyleCop.Analyzers.CodeFixes.dll
        ├── 3e67c05e-a673-4410-99a8-dee96b503071
        │   └── Microsoft.CodeAnalysis.NetAnalyzers.dll
        ├── 4583e009-12ca-4440-b8a1-6cd53d688826
        │   └── Microsoft.CodeAnalysis.CSharp.NetAnalyzers.dll
        ├── 4ae1e6af-ef06-4d30-bd1b-a750d9da15af
        │   └── Microsoft.Interop.LibraryImportGenerator.dll
        ├── 4d70db62-395d-4b20-9d8b-14b6e07b2897
        │   └── AsyncFixer.dll
        ├── 4f12f1d3-92e5-4b0d-9124-ef836cbd2389
        │   └── Microsoft.AspNetCore.App.Analyzers.dll
        ├── 5abd8b85-009b-4128-be6e-76668ad4e60a
        │   └── Microsoft.AspNetCore.App.CodeFixes.dll
        ├── 64888b1e-8094-4cf7-ac4a-3694db9cff2b
        │   └── Microsoft.EntityFrameworkCore.Analyzers.dll
        ├── 708d1fb6-e9a0-4803-bf65-0538c56decc4
        │   └── Microsoft.CodeAnalysis.Razor.Compiler.dll
        ├── 82064f6b-9278-4bab-a75b-41798fb88456
        │   └── Microsoft.Interop.JavaScript.JSImportGenerator.dll
        ├── 8c139eb1-a8e5-434e-bd11-c5f9bf624162
        │   └── Lindhart.Analyser.MissingAwaitWarning.dll
        ├── 8fdeef92-d6ee-472b-a264-e236956e3f0f
        │   └── Microsoft.CodeAnalysis.BannedApiAnalyzers.dll
        ├── 93c392bd-329d-49d9-975e-7736f909a61d
        │   └── System.Text.RegularExpressions.Generator.dll
        ├── 96520969-f53e-4ec2-abee-d215892ddf56
        │   └── NSubstitute.Analyzers.Shared.dll
        ├── b16d67c9-b8f7-44f1-a679-211dacfee914
        │   └── System.Text.Json.SourceGeneration.dll
        ├── b1c0ba86-da49-4b27-9c94-5c55bc6d4787
        │   └── Microsoft.AspNetCore.Analyzers.dll
        ├── b2c91589-0af9-4a97-ad8e-dee05d734783
        │   └── NSubstitute.Analyzers.CSharp.dll
        ├── b597c20c-1796-48ff-9e5b-fdb713983a15
        │   └── StyleCop.Analyzers.dll
        ├── c39ef408-839c-4af9-9a74-852e6094b653
        │   └── Microsoft.Extensions.Logging.Generators.dll
        ├── c3b7ab10-80ec-4e4d-97ac-52879f4bbceb
        │   └── Microsoft.AspNetCore.Mvc.Analyzers.dll
        ├── c772f166-cce3-46b3-a396-9e180f98ed4a
        │   └── IDisposableAnalyzers.dll
        ├── d1a06d15-624f-4344-95d3-525ea286bef3
        │   └── Microsoft.AspNetCore.Components.Analyzers.dll
        ├── d27ab32d-9ab6-4d36-871f-81a4635a4db1
        │   └── Microsoft.CodeAnalysis.CSharp.CodeStyle.Fixes.dll
        ├── d2fc6e9d-565a-43fa-934d-5f0bbc75750c
        │   └── Microsoft.Extensions.Logging.Generators.dll
        ├── e5632702-192c-43a7-ad8e-5453caa75b08
        │   └── Microsoft.Interop.SourceGeneration.dll
        ├── ecc0fd13-0748-46a0-8605-15ce054846e1
        │   └── SerilogAnalyzer.dll
        ├── f9fe9f28-376f-4a3f-968b-7a24ef465f83
        │   └── ToStringWithoutOverrideAnalyzer.dll
        └── fe834abc-d13c-4aba-ad8e-a8f06f15e6d2
            └── Microsoft.CodeAnalysis.CSharp.BannedApiAnalyzers.dll

I hope if others see this issue they help by showing what sort of stuff exists in their /tmp/. This is probably the result of many dotnet-related tools, so might take a while to track down everything.

Thanks for being receptive to the issue.

CyrusNajmabadi commented 2 months ago

AnalyzerAssemblyLoader is us (roslyn). It could be the compiler, but it's also possible it's the IDE code. Tagging @jaredpar and @dibarbet .

The rest do seem to be Dotnet/MSbuild. However, it's possible it's still us wrt to calling into them and not giving them a good /tmp location (i genuinely don't know). Jared is likely a good starting point here given that vbcscompiler is def us.

jaredpar commented 2 months ago

AnalyzerAssemblyLoader is us (roslyn). It could be the compiler, but it's also possible it's the IDE code.

This is for Linux correct? If so this is very likely DevKit as the compiler doesn't use /tmp/ anymore. Our Linux strategy is to do stream based loading vs. shadow copying. The IDE should definitely be using that route as it's a much simpler story.

@dibarbet sync up with me and I'll help you get this sorted. Should be a very straight forward change.

dibarbet commented 1 month ago

Resolving this issue - https://github.com/dotnet/roslyn/pull/73809 removes the cases where we were shadow copying analyzers to /tmp on linux. This will be in an upcoming prerelease of the C# extension (most likely released next Tuesday).

The others aren't controlled directly by us and mostly look like named pipes as you mention. For MSBuildTempa or NuGetScratcha you'll likely need to followup with those teams

lonix1 commented 1 month ago

Thanks I'll follow up with other teams as you proposed.

But does someone know who is responsible for those named pipes?