Add support for analysis on Windows path longer than 260 characters

bcouavoux commented 3 years ago

Description

Hello, We have an error during our Prepare analysis on SonarCloud. The problem happened today and we don't understand where that can come from. Our pipeline is executed on our private Agent.

Repro steps

Our yaml pipeline :

task: SonarSource.sonarcloud.14d9cde6-c1da-4d55-aa01-2965cd301255.SonarCloudPrepare@1 displayName: 'Prepare analysis on SonarCloud' inputs: SonarCloud: SonarCloud organization: OURORGA projectKey: '$(Sonar.Project.Key)' projectName: '$(Sonar.Project.Name)' projectVersion: '$(Sonar.Project.Version)' extraProperties: | sonar.links.homepage=$(Build.Repository.Uri)?path=%2FREADME.md&_a=preview sonar.links.ci=$(System.TeamFoundationCollectionUri)$(System.TeamProject)/_build?definitionId=$(System.DefinitionId) sonar.links.issue=$(System.TeamFoundationCollectionUri)$(System.TeamProject)/_boards sonar.links.scm=$(Build.Repository.Uri) sonar.test.inclusions=$(Repository.Subfolders.Tests)/*/Test.cs,$(Repository.Subfolders.Sources)/OURDIR/$(Repository.Subfolders.Tests)/*/.test.tsx sonar.testExecutionReportPaths=$(Build.SourcesDirectory)\$(Repository.Subfolders.Artifacts)\JestSonar\test\test-report.xml sonar.javascript.lcov.reportPaths=$(Build.SourcesDirectory)\$(Repository.Subfolders.Artifacts)\JestSonar\coverage\lcov.info sonar.cs.vscoveragexml.reportsPaths=$(Agent.BuildDirectory)\TestResults sonar.c.file.suffixes=- sonar.cpp.file.suffixes=- sonar.objc.file.suffixes=-
task: VSBuild@1 displayName: 'Build solution' inputs: solution: '*.sln' msbuildArgs: '/m /nodereuse:false' platform: '$(buildPlatform)' configuration: '$(buildConfiguration)'
task: SonarSource.sonarcloud.ce096e50-6155-4de8-8800-4221aaeed4a1.SonarCloudAnalyze@1 displayName: 'Run Code Analysis' continueOnError: true

Actual behavior

The error :

Starting: Prepare analysis on SonarCloud Task : Prepare Analysis Configuration Description : Prepare SonarCloud analysis configuration Version : 1.16.0 Author : sonarsource Help : Version: 1.16.0. More Information SYSTEMVSSCONNECTION exists true F:\agent-03-A_work_tasks\SonarCloudPrepare_14d9cde6-c1da-4d55-aa01-2965cd301255\1.16.0\classic-sonar-scanner-msbuild\SonarScanner.MSBuild.exe begin /k:PROJECT /o:OURORGA SonarScanner for MSBuild 5.0.4 Using the .NET Framework version of the Scanner for MSBuild Pre-processing started. Preparing working directories... 16:26:58.064 Updating build integration targets... [error]16:26:58.094 Failed to create an empty directory 'F:\vm-alm-agent-03-A_work\1977.sonarqube\out'. Please check that there are no open or read-only files in the directory and that you have the necessary read/write permissions. Detailed error message: Could not find a part of the path 'm_alm_agent_03_Awork_1977_xxx_xxxx_xxxxxxxxxxxxx.ucfg'. 16:26:58.094 Failed to create an empty directory 'F:\vm-alm-agent-03-A_work\1977.sonarqube\out'. Please check that there are no open or read-only files in the directory and that you have the necessary read/write permissions. Detailed error message: Could not find a part of the path 'm_alm_agent_03_Awork_1977_s_xxxxx_xxxxxxxx_xxxxxxxxxxxxxx.ucfg'. [error]16:26:58.094 Pre-processing failed. Exit code: 1 16:26:58.094 Pre-processing failed. Exit code: 1 [error]The process 'F:\agent-03-A_work_tasks\SonarCloudPrepare_14d9cde6-c1da-4d55-aa01-2965cd301255\1.16.0\classic-sonar-scanner-msbuild\SonarScanner.MSBuild.exe' failed with exit code 1 Finishing: Prepare analysis on SonarCloud

the problematic file on our private agent :

Our agent has the admin privileg on the build directory and everything worked fine before today We have seen that this may be related to dotnet and msbuild reuse of nodes that were left running by a previous multi-threaded build. So we tried with the / nodereuse parameter: false on MSbuild but same error.

nunzo-dev commented 3 years ago

huum I have the same problem and I don't understand what's not working :(

supertom78 commented 3 years ago

I am working with @bcouavoux We have solved the issue by using the property clean from the workspace with option "all" (https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema%2Cparameter-schema)

I don't know why but by .sonarquble temp directory is deleted without error

johan-v-r commented 3 years ago

Same issue here, started around the same time. Clean workspace worked as suggested by @supertom78 thanks!

My error message is similar in that it can't find the file to delete, but the whole filepath looks like it's replaced with underscores:

Detailed error message: Could not find a part of the path 'C__vsts_agent_win_x64_2_173_0__work_8_s_xxxxxxx

brianveltman commented 3 years ago

We're having the same issue with Sonar scanner for Azure DevOps 1.21.0.

pavel-mikula-sonarsource commented 3 years ago

Hi @bcouavoux, I see two suspicious things in your logs:

16:26:58.064 Updating build integration targets... [error]16:26:58.094 Failed to create an empty directory 'F:\vm-alm-agent-03-A_work\1977.sonarqube\out'. Please check that there are no open or read-only files in the directory and that you have the necessary read/write permissions.

If 1977 is your Scanner working directory, than you should have \1977\.sonarqube as a working folder. Do you have some path configuration as part of your build setup?

Detailed error message: Could not find a part of the path 'm_alm_agent_03_A__work_1977_xxx_xxxx_xxxxxxxxxxxxx.ucfg'.

UCFG files are generated during build step and processed during scanner end step. They are not created nor processed during the begin step. Are you sure you don't have two pipelines running from the same directory at once?

iamrahul127 commented 2 years ago

Hi @pavel-mikula-sonarsource, We are also getting this error randomly on some builds. We managed to get it working by using PowerShell script to delete 'out' folder. Below explanation should clarify why behavior is random.
I think issue is popping up only if file path is more than 260 characters. In my scenario file name is 274 characters long. After digging up more I found code which is throwing error. Code snippet is from \sonar-scanner-msbuild\src\SonarScanner.MSBuild.Common\Utilities.cs. Directory.Delete(directory, true) throws exception if long path is encountered.

Could you please take a look into this?

1. public static void EnsureEmptyDirectory(string directory, ILogger logger)
2.         {
3.             if (string.IsNullOrWhiteSpace(directory))
4.             {
5.                 throw new ArgumentNullException(nameof(directory));
6.             }
7.             if (logger == null)
8.             {
9.                 throw new ArgumentNullException(nameof(logger));
10.             }
11. 
12.             if (Directory.Exists(directory))
13.             {
14.                 logger.LogDebug(Resources.MSG_DeletingDirectory, directory);
15.                 Directory.Delete(directory, true);
16.             }
17.             logger.LogDebug(Resources.MSG_CreatingDirectory, directory);
18.             Directory.CreateDirectory(directory);
19.         }

Verbose logs:

Starting: Prepare analysis on SonarQube

Task : Prepare Analysis Configuration Description : Prepare SonarQube analysis configuration Version : 4.23.1 Author : sonarsource Help : Version: 4.23.1. More Information

SYSTEMVSSCONNECTION exists true F:\A\_work_tasks\SonarQubePrepare_15b84ca1-b62f-4a2a-a403-89b77a063157\4.23.1\classic-sonar-scanner-msbuild\SonarScanner.MSBuild.exe begin /k:WE.Education.ChildSchedule SonarScanner for MSBuild 5.3.1 Using the .NET Framework version of the Scanner for MSBuild Pre-processing started. Preparing working directories... 11:06:15.973 11:06:15.958 Loading analysis properties from F:\A\_work_tasks\SonarQubePrepare_15b84ca1-b62f-4a2a-a403-89b77a063157\4.23.1\classic-sonar-scanner-msbuild\SonarQube.Analysis.xml 11:06:15.973 11:06:15.973 sonar.verbose=true was specified - setting the log verbosity to 'Debug' 11:06:15.973 Updating build integration targets... 11:06:15.989 The file SonarQube.Integration.ImportBefore.targets is up to date at C:\Users\azureuser\AppData\Local\Microsoft\MSBuild\4.0\Microsoft.Common.targets\ImportBefore 11:06:15.989 The file SonarQube.Integration.ImportBefore.targets is up to date at C:\Users\azureuser\AppData\Local\Microsoft\MSBuild\10.0\Microsoft.Common.targets\ImportBefore 11:06:15.989 The file SonarQube.Integration.ImportBefore.targets is up to date at C:\Users\azureuser\AppData\Local\Microsoft\MSBuild\11.0\Microsoft.Common.targets\ImportBefore 11:06:15.989 The file SonarQube.Integration.ImportBefore.targets is up to date at C:\Users\azureuser\AppData\Local\Microsoft\MSBuild\12.0\Microsoft.Common.targets\ImportBefore 11:06:15.989 The file SonarQube.Integration.ImportBefore.targets is up to date at C:\Users\azureuser\AppData\Local\Microsoft\MSBuild\14.0\Microsoft.Common.targets\ImportBefore 11:06:15.989 The file SonarQube.Integration.ImportBefore.targets is up to date at C:\Users\azureuser\AppData\Local\Microsoft\MSBuild\15.0\Microsoft.Common.targets\ImportBefore 11:06:15.989 The file SonarQube.Integration.ImportBefore.targets is up to date at C:\Users\azureuser\AppData\Local\Microsoft\MSBuild\Current\Microsoft.Common.targets\ImportBefore 11:06:15.989 Installed SonarQube.Integration.targets to F:\A\_work\276.sonarqube\bin\targets 11:06:15.989 Creating config and output folders... 11:06:15.989 Creating directory: F:\A\_work\276.sonarqube\conf 11:06:15.989 Removing the existing directory: F:\A\_work\276.sonarqube\out

[error]11:06:15.989 Failed to create an empty directory 'F:\A\_work\276.sonarqube\out'. Please check that there are no open or read-only files in the directory and that you have the necessary read/write permissions.

Detailed error message: Could not find a part of the path '12A_B01__work_941_s_Modules_XXXXXXXXXXXXXX_Web_XX_XXXXXXXXX_XXXXXXX_XXXXXXXXXXXXX_Scripts_XXXXXXXXXXXXXXXXXXX_XXXXXXXXXXX_XXXXXXXX_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX_js_136_9_FD_470_XXXXXXXXXXXXXXXXXGetDates.ucfg'. 11:06:15.989 Failed to create an empty directory 'F:\A\_work\276.sonarqube\out'. Please check that there are no open or read-only files in the directory and that you have the necessary read/write permissions.

pavel-mikula-sonarsource commented 2 years ago

Hi @iamrahul127,

Thank you for taking a look into this. I confirm that the problem is due to the long path.

cc @tom-howlett-sonarsource to consider priority of this.

kokke commented 2 years ago

I have seen the same problem in a few C# projects lately.

I've fixed them by replacing the .NET versions of System.IO.Path, System.IO.File, System.IO.FileInfo, System.IO.Directory and System.IO.DirectoryInfo with drop-in replacements from this MIT-licensed project: https://github.com/alphaleonis/AlphaFS

From their README (emphasis mine):

The file system support in .NET is pretty good for most uses. However there are a few shortcomings, which this library tries to alleviate. The most notable deficiency of the standard .NET System.IO is the lack of support of advanced NTFS features, most notably extended length path support (eg. file/directory paths longer than 260 characters: System.IO.PathTooLongException).

As far as I can see, the problems come from some sort of path-validation, that happens in .NET before hitting the Win32 API.

The stacktraces often end with this:

Unhandled Exception: System.IO.PathTooLongException: The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.
   at System.IO.PathHelper.GetFullPathName()
   at System.IO.Path.LegacyNormalizePath(String path, Boolean fullCheck, Int32 maxPathLength, Boolean expandShortPaths)
   at System.IO.Path.GetFullPathInternal(String path)
   at System.IO.FileInfo.Init(String fileName, Boolean checkHost)

... or variations thereof (System.IO.Path, System.IO.File, System.IO.FileInfo, System.IO.Directory and System.IO.DirectoryInfo)

Could this be a workable fix for you? If so, I'd be happy to contribute a patch.

pavel-mikula-sonarsource commented 2 years ago

Hi @kokke, thank you for the insights. I think we'll prefer the more manual fix, as there should not be many places affected and mainly due to security concerns.

malu-mn commented 2 years ago

Changing the build definition as below stopped this error. In the build pipe, Get Sources step, Clean as true Clean options as All build directories.

duncanp-sonar commented 2 years ago

walterstypula commented 2 years ago

Using pinvoke version of delete file would solve this without adding any 3rd party libraries. Here's a example on how to do with with a c# class in powershell: https://devblogs.microsoft.com/scripting/weekend-scripter-remove-a-long-path-file/ However, you would need get all files in the directory structure via [System.IO.Directory]::EnumerateFiles($directory,"*", [System.IO.SearchOption]::AllDirectories) and then execute the pinvoke statement to delete each file. After the files have been deleted, you can then delete the offending directories without issue.
Migrate solution to .net framework 4.6.2?

The item below didn't work. I came in today and it was failing again. ~~Workaround: If you're running an azure-pipeline self hosted agent with .net framework 4.6.2 or greater installed. You can set the following registry setting in the attached screenshot to force long path support for all .net apps executing on the machine. MSFT Documentation: https://docs.microsoft.com/en-us/dotnet/api/system.io.pathtoolongexception Registry AppContext Documentation: https://docs.microsoft.com/en-us/dotnet/api/system.appcontext?view=netcore-3.1#appcontext-for-library-consumers~~

~~Note: You may need to set the HKLM\SOFTWARE\Wow6432Node\Microsoft.NETFramework\AppContext registry path as well, but I didn't need to.~~

kokke commented 2 years ago

Just my two cents:

Colleagues and I haven’t had any luck upgrading to newer .NET versions or with enabling the various registry-flags for Long Path Support. Even though a lot of official MS docs suggest that it should fix it 🤷‍♂️ It seems to fix some issues, but not all.

The only consistent fix for us, have been to not use the System.IO.<whatever> and e.g. replace with AlphaFS-equivalents. In other words: No amount of tweaks to the environment was able to fix this for us.

The PInvoke-solution probably also works fine. IIUC that is what the AlphaFS library does under the hood (accesses Win32 API directly).

rajesh-panchal commented 2 years ago

Hi @supertom78 - could you please explain me how can I clean the workplace? Do I need to delete .sonarqube folder manually or is there any command line arguments? Sorry I am not getting you. I am facing the same issue. Please help.

andrei-epure-sonarsource commented 2 years ago

ToDo: check whether both .NET Core and .NET Framework versions of the scanner are affected. If the .NET Core version is not affected, that would be a workaround. However this won't help the AZDO extension users.

andrei-epure-sonarsource commented 2 years ago

ToDo: test if by updating to .NET Framework 4.6.2 it fixes the problem. See comment of this

.NET before 4.6.2 tried to second-guess the OS. Since 4.6.2 the path is normalized to \\?.

If that fixes it, we'll close this as it'll be fixed later in Summer.

andrei-epure-sonarsource commented 2 years ago

An update on this ticket. I enabled Win32 long paths on my Windows 10 (Build 19044) like specified here.

I could not repro the problem with the netcore versions of the scanner (sonarscanner-msbuild-netcoreapp2.0, sonarscanner-msbuild-netcoreapp3.0, sonarscanner-msbuild-net5.0).

I could repro with the .NET Framework 4.6 version (sonarscanner-msbuild-net46).

Updating the net46 version to target .NET Framework 4.6.2 (DotnetVersions.props, ci-build.ps1 and package-artifacts.ps1) did not fix the problem.

I don't understand how this is happening because in these remarks, it is stated

In apps that run under versions of the .NET Framework prior to the .NET Framework 4.6.2, full paths must not exceed 260 characters to maintain compatibility with Windows operating systems. Any path in excess of 260 characters automatically throws a PathTooLongException.

So maybe I missed something when targeting 4.6.2 ... probably additional configuration.

Current workarounds:

use netcore version of the scanner
update the build definition as mentioned here

My repro steps:

modify the folder name of a simple .NET project to be very long
run the begin step

andrei-epure-sonarsource commented 2 years ago

After adding an app.manifest to the project, it seems that the 4.6.2 version works - https://github.com/SonarSource/sonar-scanner-msbuild/compare/andrei/long-path

Must also test to see if that works with 4.6 (as long as it's executed on 4.6.2). It might require an app.config file instead (See this).

andrei-epure-sonarsource commented 2 years ago

I've made the necessary changes mentioned in this post in #1265 and it will go to version 5.7.0 of the Scanner for .NET.

However, there are still cases which are not covered (such as MSBuild loading the necessary Tasks.dll) which might require more changes, hence I am not closing this ticket and will wait for feedback first.

pk27734 commented 2 years ago

I am still seeing this issue on SonarCloud using an on-prem build agent, while I believe it should be fixed. Is that correct?

andrei-epure-sonarsource commented 2 years ago

Hi @pk27734 , if you're using the Azure DevOps integration - we haven't updated the scanner on those yet (Actually, we did the update and reverted it because #1281) - when updating it I'll ping you here to verify. Thanks a lot for keeping an eye on this.

sebastien-marichal commented 1 year ago

I am closing the issue since it seems to be fixed and no additional feedback was provided regarding sonar-scanner-msbuild being used in the context of long paths.

SonarSource / sonar-scanner-msbuild