search

Azure / bicep

Bicep is a declarative language for describing and deploying Azure resources
MIT License
3.26k stars 754 forks source link

Azure.Bicep.MSBuild trying to use win-x64 bicep.exe on Mac #15228

Open cataggar opened 1 month ago

cataggar commented 1 month ago

We lock or dependencies and would like dotnet build -t:BicepCompile to work on Windows, Linux, and Mac. We are restoring all three native executable packages. It is just a little more disk space. The problem is that when I run it on my Mac, it is trying to run the win-x64 build of bicep.exe. Seems like a bug.

~/ms/avs/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates> dotnet build -t:BicepCompile
  Determining projects to restore...
  All projects are up-to-date for restore.
/Users/cataggar/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003: The specified task executable "/Users/cataggar/.nuget/packages/azure.bicep.commandline.win-x64/0.30.23/build/../tools/bicep.exe" could not be run. System.ComponentModel.Win32Exception (8): An error occurred trying to start process '/Users/cataggar/.nuget/packages/azure.bicep.commandline.win-x64/0.30.23/build/../tools/bicep.exe' with working directory '/Users/cataggar/ms/avs/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates'. Exec format error [/Users/cataggar/ms/avs/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
/Users/cataggar/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003:    at System.Diagnostics.Process.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, String[] envp, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec) [/Users/cataggar/ms/avs/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
/Users/cataggar/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003:    at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo) [/Users/cataggar/ms/avs/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
/Users/cataggar/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.StartToolProcess(Process proc) [/Users/cataggar/ms/avs/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
/Users/cataggar/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [/Users/cataggar/ms/avs/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
/Users/cataggar/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.Execute() [/Users/cataggar/ms/avs/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]

Build FAILED.

/Users/cataggar/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003: The specified task executable "/Users/cataggar/.nuget/packages/azure.bicep.commandline.win-x64/0.30.23/build/../tools/bicep.exe" could not be run. System.ComponentModel.Win32Exception (8): An error occurred trying to start process '/Users/cataggar/.nuget/packages/azure.bicep.commandline.win-x64/0.30.23/build/../tools/bicep.exe' with working directory '/Users/cataggar/ms/avs/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates'. Exec format error [/Users/cataggar/ms/avs/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
/Users/cataggar/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003:    at System.Diagnostics.Process.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, String[] envp, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec) [/Users/cataggar/ms/avs/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
/Users/cataggar/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003:    at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo) [/Users/cataggar/ms/avs/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
/Users/cataggar/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.StartToolProcess(Process proc) [/Users/cataggar/ms/avs/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
/Users/cataggar/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [/Users/cataggar/ms/avs/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
/Users/cataggar/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.Execute() [/Users/cataggar/ms/avs/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
    0 Warning(s)
    1 Error(s)

Time Elapsed 00:00:00.59
~/ms/avs/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates> dotnet --info
.NET SDK:
 Version:           8.0.401
 Commit:            811edcc344
 Workload version:  8.0.400-manifests.56cd0383
 MSBuild version:   17.11.4+37eb419ad

Runtime Environment:
 OS Name:     Mac OS X
 OS Version:  15.0
 OS Platform: Darwin
 RID:         osx-arm64
 Base Path:   /usr/local/share/dotnet/sdk/8.0.401/
cataggar commented 1 month ago

Here is the Templates.proj:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <Azure-Bicep-Version>0.30.23</Azure-Bicep-Version>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Azure.Bicep.CommandLine.linux-x64" Version="$(Azure-Bicep-Version)" />
    <PackageReference Include="Azure.Bicep.CommandLine.osx-arm64" Version="$(Azure-Bicep-Version)" />
    <PackageReference Include="Azure.Bicep.CommandLine.win-x64" Version="$(Azure-Bicep-Version)" />
    <PackageReference Include="Azure.Bicep.MSBuild" Version="$(Azure-Bicep-Version)" />
  </ItemGroup>
  <ItemGroup>
    <Bicep Include="**/*.bicep" OutputFile="%(RecursiveDir)/%(FileName)-template.json" />
  </ItemGroup>
</Project>

If I remove Azure.Bicep.CommandLine.win-x64, it works, but I need that entry.

cataggar commented 1 month ago

It also fails when run on Linux in CI:

/root/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003: The specified task executable "/root/.nuget/packages/azure.bicep.commandline.win-x64/0.30.23/build/../tools/bicep.exe" could not be run. System.ComponentModel.Win32Exception (8): An error occurred trying to start process '/root/.nuget/packages/azure.bicep.commandline.win-x64/0.30.23/build/../tools/bicep.exe' with working directory '/__w/1/s/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates'. Exec format error [/__w/1/s/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
/root/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003:    at System.Diagnostics.Process.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, String[] envp, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec) [/__w/1/s/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
/root/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003:    at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo) [/__w/1/s/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
/root/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.StartToolProcess(Process proc) [/__w/1/s/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
/root/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [/__w/1/s/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
/root/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.Execute() [/__w/1/s/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
cataggar commented 1 month ago

On Linux, it tries and fails to use osx-arm64 when it and linux-x64 are restored.

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <Azure-Bicep-Version>0.30.23</Azure-Bicep-Version>
    <!-- Disable lock files as workaround for https://github.com/Azure/bicep/issues/15228 -->
    <RestorePackagesWithLockFile>false</RestorePackagesWithLockFile>
    <RestoreLockedMode>false</RestoreLockedMode>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Azure.Bicep.CommandLine.linux-x64" Version="$(Azure-Bicep-Version)" Condition="'$(OS)' == 'UNIX'" />
    <PackageReference Include="Azure.Bicep.CommandLine.osx-arm64" Version="$(Azure-Bicep-Version)" Condition="'$(OS)' == 'UNIX'" />
    <PackageReference Include="Azure.Bicep.CommandLine.win-x64" Version="$(Azure-Bicep-Version)" Condition="'$(OS)' == 'WINDOWS_NT'" />
    <PackageReference Include="Azure.Bicep.MSBuild" Version="$(Azure-Bicep-Version)" />
  </ItemGroup>
  <ItemGroup>
    <Bicep Include="**/*.bicep" OutputFile="%(RecursiveDir)/%(FileName)-template.json" />
  </ItemGroup>
</Project>
/root/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003: The specified task executable "/root/.nuget/packages/azure.bicep.commandline.osx-arm64/0.30.23/build/../tools/bicep" could not be run. System.ComponentModel.Win32Exception (8): An error occurred trying to start process '/root/.nuget/packages/azure.bicep.commandline.osx-arm64/0.30.23/build/../tools/bicep' with working directory '/__w/1/s/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates'. Exec format error [/__w/1/s/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
/root/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003:    at System.Diagnostics.Process.ForkAndExecProcess(ProcessStartInfo startInfo, String resolvedFilename, String[] argv, String[] envp, String cwd, Boolean setCredentials, UInt32 userId, UInt32 groupId, UInt32[] groups, Int32& stdinFd, Int32& stdoutFd, Int32& stderrFd, Boolean usesTerminal, Boolean throwOnNoExec) [/__w/1/s/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
/root/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003:    at System.Diagnostics.Process.StartCore(ProcessStartInfo startInfo) [/__w/1/s/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
/root/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.StartToolProcess(Process proc) [/__w/1/s/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
/root/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.ExecuteTool(String pathToTool, String responseFileCommands, String commandLineCommands) [/__w/1/s/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
/root/.nuget/packages/azure.bicep.msbuild/0.30.23/build/Azure.Bicep.MSBuild.targets(66,5): error MSB6003:    at Microsoft.Build.Utilities.ToolTask.Execute() [/__w/1/s/src/avs-releases/ev2/ev2/ServiceGroupRoot/Templates/Templates.proj]
majastrz commented 1 month ago

The Azure.Bicep.CommandLine.* packages currently assume that only one flavor is referenced by a project so either the first or last one wins. I think we should be able to do our own OS detection and only run the logic from the package that matches the architecture of the machine running the build.

Until we implement this fix, the best workaround is to set the BicepPath MSBuild property based on the OS you're running on.

majastrz commented 1 month ago

@cataggar Does the workaround I shared help with this?

cataggar commented 1 month ago

@majastrz. The workaround was a bit more complicated than that. You can see in internal PR https://dev.azure.com/msazure/One/_git/Azure-Dedicated-AVS/pullrequest/10976497 . I had to disabled lock files and support for osx. We would like to be able to use lock files and have it work on Windows, Linux, and Mac.

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <TargetFramework>net8.0</TargetFramework>
    <Azure-Bicep-Version>0.30.23</Azure-Bicep-Version>
    <!-- Disable lock files as workaround for https://github.com/Azure/bicep/issues/15228 -->
    <RestorePackagesWithLockFile>false</RestorePackagesWithLockFile>
    <RestoreLockedMode>false</RestoreLockedMode>
  </PropertyGroup>
  <ItemGroup>
    <PackageReference Include="Azure.Bicep.CommandLine.linux-x64" Version="$(Azure-Bicep-Version)" Condition="'$(OS)' == 'UNIX'" />
    <!-- Also disable osx-arm64 for same workaround -->
    <!-- <PackageReference Include="Azure.Bicep.CommandLine.osx-arm64" Version="$(Azure-Bicep-Version)" Condition="'$(OS)' == 'UNIX'" /> -->
    <PackageReference Include="Azure.Bicep.CommandLine.win-x64" Version="$(Azure-Bicep-Version)" Condition="'$(OS)' == 'WINDOWS_NT'" />
    <PackageReference Include="Azure.Bicep.MSBuild" Version="$(Azure-Bicep-Version)" />
  </ItemGroup>
  <ItemGroup>
    <Bicep Include="**/*.bicep" OutputFile="%(RecursiveDir)/%(FileName).json" />
  </ItemGroup>
</Project>
majastrz commented 1 month ago

I looked at the internal PR and that is definitely not a great experience. The platform/arch specific packages are basically just a way to set BicepPath. My fix will make the packages skip their logic unless the OS matches the package itself.