search

RicoSuter / NSwag

The Swagger/OpenAPI toolchain for .NET, ASP.NET Core and TypeScript.
http://NSwag.org
MIT License
6.75k stars 1.29k forks source link

Intermittent NSwag failure on .NET 8.0 #4998

Open WenningQiu opened 1 week ago

WenningQiu commented 1 week ago

My application is seeing intermittent build failure after being migrated to .NET 8.0 with NSwag.MSBuild upgraded to 14.1.0. So far we have been seeing the error only when the application is built by Azure DevOps pipeline and not when built locally.

The error seems to be started with a file access error: "The process cannot access the file 'D:\csg-azure-agent_work\1241\s\Bin\Debug\net8.0-windows\BillComposer.BL.deps.json' because it is being used by another process. [D:\csg-azure-agent_work\1241\s\src\BillComposer.BL\BillComposer.BL.csproj]".

The file access error has always been on the same file in the failed builds. I am not sure why NSwag needs to access that file in Debug folder when the command file specifies Release build: "dotnet "D:\Cache.nuget\packages\nswag.msbuild\14.1.0\buildTransitive../tools/Net80/dotnet-nswag.dll" run nswag.json /variables:Configuration=Release"


C:\Program Files\dotnet\sdk\8.0.304\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(284,5): error MSB4018: at Microsoft.Win32.SafeHandles.SafeFileHandle.Open(String fullPath, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode) [D:\csg-azure-agent\_work\1241\s\src\BillComposer.BL\BillComposer.BL.csproj] [D:\csg-azure-agent\_work\1241\s\src\BillComposer.API.NSwag\BillComposer.API.NSwag.csproj]
##[error]C:\Program Files\dotnet\sdk\8.0.304\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(284,5): Error MSB4018: at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode) [D:\csg-azure-agent\_work\1241\s\src\BillComposer.BL\BillComposer.BL.csproj]
C:\Program Files\dotnet\sdk\8.0.304\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(284,5): error MSB4018: at System.IO.Strategies.OSFileStreamStrategy..ctor(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode) [D:\csg-azure-agent\_work\1241\s\src\BillComposer.BL\BillComposer.BL.csproj] [D:\csg-azure-agent\_work\1241\s\src\BillComposer.API.NSwag\BillComposer.API.NSwag.csproj]
##[error]C:\Program Files\dotnet\sdk\8.0.304\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(284,5): Error MSB4018: at System.IO.Strategies.FileStreamHelpers.ChooseStrategyCore(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode) [D:\csg-azure-agent\_work\1241\s\src\BillComposer.BL\BillComposer.BL.csproj]
C:\Program Files\dotnet\sdk\8.0.304\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(284,5): error MSB4018: at System.IO.Strategies.FileStreamHelpers.ChooseStrategyCore(String path, FileMode mode, FileAccess access, FileShare share, FileOptions options, Int64 preallocationSize, Nullable`1 unixCreateMode) [D:\csg-azure-agent\_work\1241\s\src\BillComposer.BL\BillComposer.BL.csproj] [D:\csg-azure-agent\_work\1241\s\src\BillComposer.API.NSwag\BillComposer.API.NSwag.csproj]
##[error]C:\Program Files\dotnet\sdk\8.0.304\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(284,5): Error MSB4018: at System.IO.File.Create(String path) [D:\csg-azure-agent\_work\1241\s\src\BillComposer.BL\BillComposer.BL.csproj]
C:\Program Files\dotnet\sdk\8.0.304\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(284,5): error MSB4018: at System.IO.File.Create(String path) [D:\csg-azure-agent\_work\1241\s\src\BillComposer.BL\BillComposer.BL.csproj] [D:\csg-azure-agent\_work\1241\s\src\BillComposer.API.NSwag\BillComposer.API.NSwag.csproj]
##[error]C:\Program Files\dotnet\sdk\8.0.304\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(284,5): Error MSB4018: at Microsoft.NET.Build.Tasks.GenerateDepsFile.WriteDepsFile(String depsFilePath) [D:\csg-azure-agent\_work\1241\s\src\BillComposer.BL\BillComposer.BL.csproj]
C:\Program Files\dotnet\sdk\8.0.304\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(284,5): error MSB4018: at Microsoft.NET.Build.Tasks.GenerateDepsFile.WriteDepsFile(String depsFilePath) [D:\csg-azure-agent\_work\1241\s\src\BillComposer.BL\BillComposer.BL.csproj] [D:\csg-azure-agent\_work\1241\s\src\BillComposer.API.NSwag\BillComposer.API.NSwag.csproj]
##[error]C:\Program Files\dotnet\sdk\8.0.304\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(284,5): Error MSB4018: at Microsoft.NET.Build.Tasks.TaskBase.Execute() [D:\csg-azure-agent\_work\1241\s\src\BillComposer.BL\BillComposer.BL.csproj]
C:\Program Files\dotnet\sdk\8.0.304\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(284,5): error MSB4018: at Microsoft.NET.Build.Tasks.TaskBase.Execute() [D:\csg-azure-agent\_work\1241\s\src\BillComposer.BL\BillComposer.BL.csproj] [D:\csg-azure-agent\_work\1241\s\src\BillComposer.API.NSwag\BillComposer.API.NSwag.csproj]
##[error]C:\Program Files\dotnet\sdk\8.0.304\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(284,5): Error MSB4018: at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [D:\csg-azure-agent\_work\1241\s\src\BillComposer.BL\BillComposer.BL.csproj]
C:\Program Files\dotnet\sdk\8.0.304\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(284,5): error MSB4018: at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [D:\csg-azure-agent\_work\1241\s\src\BillComposer.BL\BillComposer.BL.csproj] [D:\csg-azure-agent\_work\1241\s\src\BillComposer.API.NSwag\BillComposer.API.NSwag.csproj]
##[error]C:\Program Files\dotnet\sdk\8.0.304\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(284,5): Error MSB4018: at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [D:\csg-azure-agent\_work\1241\s\src\BillComposer.BL\BillComposer.BL.csproj]
C:\Program Files\dotnet\sdk\8.0.304\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.Sdk.targets(284,5): error MSB4018: at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [D:\csg-azure-agent\_work\1241\s\src\BillComposer.BL\BillComposer.BL.csproj] [D:\csg-azure-agent\_work\1241\s\src\BillComposer.API.NSwag\BillComposer.API.NSwag.csproj]
  System.InvalidOperationException: Unable to retrieve project metadata. Ensure it's an MSBuild-based .NET Core project.If you're using custom BaseIntermediateOutputPath or MSBuildProjectExtensionsPath values, Use the --msbuildprojectextensionspath option.
     at NSwag.Commands.Generation.AspNetCore.ProjectMetadata.ReadUsingMsBuildTargets(List`1 args, String file, String buildExtensionsDir, IConsoleHost console) in /_/src/NSwag.Commands/Commands/Generation/AspNetCore/ProjectMetadata.cs:line 160
     at NSwag.Commands.Generation.AspNetCore.ProjectMetadata.GetProjectMetadata(String file, String buildExtensionsDir, String framework, String configuration, String runtime, Boolean noBuild, String outputPath, IConsoleHost console) in /_/src/NSwag.Commands/Commands/Generation/AspNetCore/ProjectMetadata.cs:line 92
     at NSwag.Commands.Generation.AspNetCore.AspNetCoreToOpenApiCommand.RunAsync(CommandLineProcessor processor, IConsoleHost host) in /_/src/NSwag.Commands/Commands/Generation/AspNetCore/AspNetCoreToOpenApiCommand.cs:line 70
     at NSwag.Commands.NSwagDocumentBase.GenerateSwaggerDocumentAsync() in /_/src/NSwag.Commands/NSwagDocumentBase.cs:line 270
     at NSwag.Commands.NSwagDocument.ExecuteAsync() in /_/src/NSwag.Commands/NSwagDocument.cs:line 67
     at NSwag.Commands.Document.ExecuteDocumentCommand.ExecuteDocumentAsync(IConsoleHost host, String filePath) in /_/src/NSwag.Commands/Commands/Document/ExecuteDocumentCommand.cs:line 76
     at NSwag.Commands.Document.ExecuteDocumentCommand.RunAsync(CommandLineProcessor processor, IConsoleHost host) in /_/src/NSwag.Commands/Commands/Document/ExecuteDocumentCommand.cs:line 33
     at NConsole.CommandLineProcessor.ProcessSingleAsync(String[] args, Object input)
     at NConsole.CommandLineProcessor.ProcessAsync(String[] args, Object input)
     at NSwag.Commands.NSwagCommandProcessor.ProcessAsync(String[] args) in /_/src/NSwag.Commands/NSwagCommandProcessor.cs:line 65
##[error]src\BillComposer.API.NSwag\BillComposer.API.NSwag.csproj(46,5): Error MSB3073: The command "dotnet "D:\Cache\.nuget\packages\nswag.msbuild\14.1.0\buildTransitive\../tools/Net80/dotnet-nswag.dll" run nswag.json /variables:Configuration=Release" exited with code -1.
D:\csg-azure-agent\_work\1241\s\src\BillComposer.API.NSwag\BillComposer.API.NSwag.csproj(46,5): error MSB3073: The command "dotnet "D:\Cache\.nuget\packages\nswag.msbuild\14.1.0\buildTransitive\../tools/Net80/dotnet-nswag.dll" run nswag.json /variables:Configuration=Release" exited with code -1.
Done Building Project "D:\csg-azure-agent\_work\1241\s\src\BillComposer.API.NSwag\BillComposer.API.NSwag.csproj" (default targets) -- FAILED.
Done Building Project "D:\csg-azure-agent\_work\1241\s\src\BillComposer.API.NSwag\BillComposer.API.NSwag.csproj.metaproj" (default targets) -- FAILED.
Done Building Project "D:\csg-azure-agent\_work\1241\s\BillComposer.sln" (Clean;Build target(s)) -- FAILED.

Below are the .csproj snippet that is relavant to NSwag:

  <ItemGroup>
    <PackageReference Include="NSwag.MSBuild" Version="14.1.0">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
  </ItemGroup>
  <Target Name="NSwag" AfterTargets="PostBuildEvent">
    <Exec Command="$(NSwagExe_Net80) run nswag.json /variables:Configuration=$(Configuration)" />
  </Target>

Below is the content of nswag.json file:

{
  "runtime": "Net80",
  "defaultVariables": null,
  "documentGenerator": {
    "aspNetCoreToOpenApi": {
      "project": "../BillComposer.API/BillComposer.API.csproj",
      "msBuildProjectExtensionsPath": null,
      "configuration": null,
      "runtime": null,
      "targetFramework": null,
      "noBuild": false,
      "msBuildOutputPath": null,
      "verbose": true,
      "workingDirectory": null,
      "requireParametersWithoutDefault": false,
      "apiGroupNames": null,
      "defaultPropertyNameHandling": "Default",
      "defaultReferenceTypeNullHandling": "Null",
      "defaultDictionaryValueReferenceTypeNullHandling": "NotNull",
      "defaultResponseReferenceTypeNullHandling": "NotNull",
      "generateOriginalParameterNames": true,
      "defaultEnumHandling": "Integer",
      "flattenInheritanceHierarchy": false,
      "generateKnownTypes": true,
      "generateEnumMappingDescription": false,
      "generateXmlObjects": false,
      "generateAbstractProperties": false,
      "generateAbstractSchemas": true,
      "ignoreObsoleteProperties": false,
      "allowReferencesWithProperties": false,
      "useXmlDocumentation": true,
      "resolveExternalXmlDocumentation": true,
      "excludedTypeNames": [],
      "serviceHost": null,
      "serviceBasePath": null,
      "serviceSchemes": [],
      "infoTitle": "My Title",
      "infoDescription": null,
      "infoVersion": "1.0.0",
      "documentTemplate": null,
      "documentProcessorTypes": [],
      "operationProcessorTypes": [],
      "typeNameGeneratorType": null,
      "schemaNameGeneratorType": null,
      "contractResolverType": null,
      "serializerSettingsType": null,
      "useDocumentProvider": true,
      "documentName": "v1",
      "aspNetCoreEnvironment": null,
      "createWebHostBuilderMethod": null,
      "startupType": null,
      "allowNullableBodyParameters": true,
      "useHttpAttributeNameAsOperationId": false,
      "output": null,
      "outputType": "Swagger2",
      "newLineBehavior": "Auto",
      "assemblyPaths": [],
      "assemblyConfig": null,
      "referencePaths": [],
      "useNuGetCache": false
    }
  },
  "codeGenerators": {
    "openApiToTypeScriptClient": {
      "className": "{controller}Client",
      "moduleName": "",
      "namespace": "",
      "typeScriptVersion": 4.3,
      "template": "Fetch",
      "promiseType": "Promise",
      "httpClass": "HttpClient",
      "withCredentials": false,
      "useSingletonProvider": false,
      "injectionTokenType": "OpaqueToken",
      "rxJsVersion": 6.0,
      "dateTimeType": "Date",
      "nullValue": "Undefined",
      "generateClientClasses": true,
      "generateClientInterfaces": false,
      "generateOptionalParameters": false,
      "exportTypes": true,
      "wrapDtoExceptions": false,
      "exceptionClass": "ApiException",
      "clientBaseClass": null,
      "wrapResponses": false,
      "wrapResponseMethods": [],
      "generateResponseClasses": true,
      "responseClass": "SwaggerResponse",
      "protectedMethods": [],
      "configurationClass": null,
      "useTransformOptionsMethod": false,
      "useTransformResultMethod": false,
      "generateDtoTypes": true,
      "operationGenerationMode": "MultipleClientsFromOperationId",
      "markOptionalProperties": true,
      "generateCloneMethod": false,
      "typeStyle": "Class",
      "enumStyle": "Enum",
      "useLeafType": false,
      "classTypes": [],
      "extendedClasses": [],
      "extensionCode": null,
      "generateDefaultValues": true,
      "excludedTypeNames": [],
      "excludedParameterNames": [],
      "handleReferences": false,
      "generateConstructorInterface": true,
      "convertConstructorInterfaceData": false,
      "importRequiredTypes": true,
      "useGetBaseUrlMethod": false,
      "baseUrlTokenName": "API_BASE_URL",
      "queryNullValue": "",
      "useAbortSignal": false,
      "inlineNamedDictionaries": false,
      "inlineNamedAny": false,
      "includeHttpContext": false,
      "templateDirectory": null,
      "typeNameGeneratorType": null,
      "propertyNameGeneratorType": null,
      "enumNameGeneratorType": null,
      "serviceHost": null,
      "serviceSchemes": null,
      "output": "../BillComposer.UI/src/api/client.ts",
      "newLineBehavior": "Auto"
    }
  }
}
WenningQiu commented 6 days ago

Update - I see the error on my local machine too, although much less frequently than on Azure DevOps pipeline.

  1. Why does NSwag drives Debug build even with "Configuration=Release"?
  2. Does NSwag kicks off parallel build tasks that may result in file contention?
NSwag:
  dotnet "C:\Users\qiuw01\.nuget\packages\nswag.msbuild\14.1.0\buildTransitive\../tools/Net80/dotnet-nswag.dll" run nswag.json /variables:Configuration=Release
  NSwag command line tool for .NET Core Net80, toolchain v14.1.0.0 (NJsonSchema v11.0.2.0 (Newtonsoft.Json v13.0.0.0))
  Visit http://NSwag.org for more information.
  NSwag bin directory: C:\Users\qiuw01\.nuget\packages\nswag.msbuild\14.1.0\tools\Net80

  Executing file 'nswag.json' with variables 'Configuration=Release'...
CSC : error CS2012: Cannot open 'C:\work\git\os_system\BillComposer\src\BillComposer.API\obj\Debug\net8.0-windows\BillComposer.API.dll' for writing -- 'The process cannot access the file 'C:\work\git\os_system\
BillComposer\src\BillComposer.API\obj\Debug\net8.0-windows\BillComposer.API.dll' because it is being used by another process.' [C:\work\git\os_system\BillComposer\src\BillComposer.API\BillComposer.API.csproj] [
C:\work\git\os_system\BillComposer\src\BillComposer.API.NSwag\BillComposer.API.NSwag.csproj]
  System.InvalidOperationException: Unable to retrieve project metadata. Ensure it's an MSBuild-based .NET Core project.If you're using custom BaseIntermediateOutputPath or MSBuildProjectExtensionsPath values,
  Use the --msbuildprojectextensionspath option.
     at NSwag.Commands.Generation.AspNetCore.ProjectMetadata.ReadUsingMsBuildTargets(List`1 args, String file, String buildExtensionsDir, IConsoleHost console) in /_/src/NSwag.Commands/Commands/Generation/AspNe
  tCore/ProjectMetadata.cs:line 160
     at NSwag.Commands.Generation.AspNetCore.ProjectMetadata.GetProjectMetadata(String file, String buildExtensionsDir, String framework, String configuration, String runtime, Boolean noBuild, String outputPath
  , IConsoleHost console) in /_/src/NSwag.Commands/Commands/Generation/AspNetCore/ProjectMetadata.cs:line 92
     at NSwag.Commands.Generation.AspNetCore.AspNetCoreToOpenApiCommand.RunAsync(CommandLineProcessor processor, IConsoleHost host) in /_/src/NSwag.Commands/Commands/Generation/AspNetCore/AspNetCoreToOpenApiCom
  mand.cs:line 70
     at NSwag.Commands.NSwagDocumentBase.GenerateSwaggerDocumentAsync() in /_/src/NSwag.Commands/NSwagDocumentBase.cs:line 270
     at NSwag.Commands.NSwagDocument.ExecuteAsync() in /_/src/NSwag.Commands/NSwagDocument.cs:line 67
     at NSwag.Commands.Document.ExecuteDocumentCommand.ExecuteDocumentAsync(IConsoleHost host, String filePath) in /_/src/NSwag.Commands/Commands/Document/ExecuteDocumentCommand.cs:line 76
     at NSwag.Commands.Document.ExecuteDocumentCommand.RunAsync(CommandLineProcessor processor, IConsoleHost host) in /_/src/NSwag.Commands/Commands/Document/ExecuteDocumentCommand.cs:line 33
     at NConsole.CommandLineProcessor.ProcessSingleAsync(String[] args, Object input)
     at NConsole.CommandLineProcessor.ProcessAsync(String[] args, Object input)
     at NSwag.Commands.NSwagCommandProcessor.ProcessAsync(String[] args) in /_/src/NSwag.Commands/NSwagCommandProcessor.cs:line 65
C:\work\git\os_system\BillComposer\src\BillComposer.API.NSwag\BillComposer.API.NSwag.csproj(69,5): error MSB3073: The command "dotnet "C:\Users\qiuw01\.nuget\packages\nswag.msbuild\14.1.0\buildTransitive\../too
ls/Net80/dotnet-nswag.dll" run nswag.json /variables:Configuration=Release" exited with code -1.
vvdb-architecture commented 3 days ago

The error happens intermittently. That is usually because of parallelism. It's a file in a project called BillComposer.BL.csproj, and your NSwag invocation is from BillComposer.API.NSwag.csproj. If no direct or indirect dependencies exist between these two projects, the builds may occur in parallel, triggering the problem.

If you can't add the former as a dependency on the latter, perhaps you can try to set the build order explicitly in Visual Studio... just a thought.

SerenaPrososki commented 22 hours ago

I added the following project dependency as suggested. I am still getting the error above that Wenning mentions.

EndProject Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "BillComposer.API.NSwag", "src\BillComposer.API.NSwag\BillComposer.API.NSwag.csproj", "{30B86CD0-25C4-4F7E-8B1F-4BAD9498DF1D}" ProjectSection(ProjectDependencies) = postProject {9DB34C69-3CB7-4FB0-B2BA-DB3F65217051} = {9DB34C69-3CB7-4FB0-B2BA-DB3F65217051} {ACE9289D-70F8-4615-B947-D0D6ED0857ED} = {ACE9289D-70F8-4615-B947-D0D6ED0857ED} EndProjectSection EndProject

MichaelSpencerJr commented 22 hours ago

(I'm a coworker of Serena and Wenning) We're trying some other changes on our end, and will come back to this ticket if we are still stuck after. Thanks again, NSwag team.

vvdb-architecture commented 14 hours ago

I don't use .sln mods, but the .csproj way

<ProjectReference Include="...\BillComposer.BL.csproj">
  <ReferenceOutputAssembly>false</ReferenceOutputAssembly>
</ProjectReference>