search

dotnet / runtime

.NET is a cross-platform runtime for cloud, mobile, desktop, and IoT apps.
https://docs.microsoft.com/dotnet/core/
MIT License
14.91k stars 4.63k forks source link

Executable from `dotnet publish` for Apple Silicon is not working. .net 6 preview 1. #49091

Closed iamphi closed 3 years ago

iamphi commented 3 years ago

The executable from dotnet publish --self-contained -r osx-arm64 does not run as expected. It fails with a kill signal like: zsh: killed

Test script:

mkdir helloworld
cd helloworld 
dotnet new console

echo "### Testing dotnet run"
dotnet run

echo "### Testing intel publish"
dotnet publish --self-contained -r osx-x64 -o intel
lipo -info intel/helloworld
./intel/helloworld                                                     

echo "### Testing arm64 publish"
dotnet publish --self-contained -r osx-arm64 -o arm64
lipo -info arm64/helloworld
./arm64/helloworld

My output:

The template "Console Application" was created successfully.

Processing post-creation actions...
Running 'dotnet restore' on /Users/phi/ws/helloworld/helloworld.csproj...
  Determining projects to restore...
  Restored /Users/phi/ws/helloworld/helloworld.csproj (in 138 ms).
Restore succeeded.

### Testing dotnet run
Hello World!
### Testing intel publish
Microsoft (R) Build Engine version 16.9.0-preview-21103-02+198f3f262 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  Restored /Users/phi/ws/helloworld/helloworld.csproj (in 207 ms).
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  helloworld -> /Users/phi/ws/helloworld/bin/Debug/net6.0/osx-x64/helloworld.dll
  helloworld -> /Users/phi/ws/helloworld/intel/
Non-fat file: intel/helloworld is architecture: x86_64
Hello World!
### Testing arm64 publish
Microsoft (R) Build Engine version 16.9.0-preview-21103-02+198f3f262 for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  Restored /Users/phi/ws/helloworld/helloworld.csproj (in 216 ms).
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  helloworld -> /Users/phi/ws/helloworld/bin/Debug/net6.0/osx-arm64/helloworld.dll
  helloworld -> /Users/phi/ws/helloworld/arm64/
Non-fat file: arm64/helloworld is architecture: arm64
zsh: killed     ./arm64/helloworld

Machine information:

% sw_vers 
ProductName:    macOS
ProductVersion: 11.1
BuildVersion:   20C69
% system_profiler -detailLevel mini SPHardwareDataType
Hardware:

    Hardware Overview:

      Model Name: Mac mini
      Model Identifier: Macmini9,1
      Chip: Apple M1
      Total Number of Cores: 8 (4 performance and 4 efficiency)
      Memory: 8 GB
      System Firmware Version: 6723.61.3
% dotnet --version
6.0.100-preview.1.21103.13
% dotnet --list-runtimes
Microsoft.AspNetCore.App 6.0.0-preview.1.21103.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 6.0.0-preview.1.21102.12 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
% dotnet --list-sdks    
6.0.100-preview.1.21103.13 [/usr/local/share/dotnet/sdk]
dotnet-issue-labeler[bot] commented 3 years ago

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

dotnet-issue-labeler[bot] commented 3 years ago

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

bholmes commented 3 years ago

I hit this as well and found that you need to sign the binary.

codesign -s _identity_ arm64/helloworld

iamphi commented 3 years ago

@bholmes Thank you. I was able to sign without an identity with: codesign -s - arm64/helloworld and helloworld was able to run on my computer. I zipped up arm64 and gave it to my colleague and they were able to run it too.

From "macOS Big Sur 11.0.1 Universal Apps Release Notes" https://developer.apple.com/documentation/macos-release-notes/macos-big-sur-11_0_1-universal-apps-release-notes

New in macOS 11 on Macs with Apple silicon, and starting in macOS Big Sur 11 beta 6, the operating system enforces that any executable must be signed before it’s allowed to run. There isn’t a specific identity requirement for this signature: a simple ad-hoc signature is sufficient. This new behavior doesn’t change the long-established policy that our users and developers can run arbitrary code on their Macs, and is designed to simplify the execution policies on Macs with Apple silicon and enable the system to better detect code modifications. This new policy doesn’t apply to translated x86 binaries running under Rosetta 2, nor does it apply to macOS 11 running on Intel-based platforms.

ghost commented 3 years ago

Tagging subscribers to this area: @vitek-karas, @agocke See info in area-owners.md if you want to be subscribed.

Issue Details
The executable from `dotnet publish --self-contained -r osx-arm64` does not run as expected. It fails with a kill signal like: `zsh: killed` Test script: ``` mkdir helloworld cd helloworld dotnet new console echo "### Testing dotnet run" dotnet run echo "### Testing intel publish" dotnet publish --self-contained -r osx-x64 -o intel lipo -info intel/helloworld ./intel/helloworld echo "### Testing arm64 publish" dotnet publish --self-contained -r osx-arm64 -o arm64 lipo -info arm64/helloworld ./arm64/helloworld ``` My output: ``` The template "Console Application" was created successfully. Processing post-creation actions... Running 'dotnet restore' on /Users/phi/ws/helloworld/helloworld.csproj... Determining projects to restore... Restored /Users/phi/ws/helloworld/helloworld.csproj (in 138 ms). Restore succeeded. ### Testing dotnet run Hello World! ### Testing intel publish Microsoft (R) Build Engine version 16.9.0-preview-21103-02+198f3f262 for .NET Copyright (C) Microsoft Corporation. All rights reserved. Determining projects to restore... Restored /Users/phi/ws/helloworld/helloworld.csproj (in 207 ms). You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview helloworld -> /Users/phi/ws/helloworld/bin/Debug/net6.0/osx-x64/helloworld.dll helloworld -> /Users/phi/ws/helloworld/intel/ Non-fat file: intel/helloworld is architecture: x86_64 Hello World! ### Testing arm64 publish Microsoft (R) Build Engine version 16.9.0-preview-21103-02+198f3f262 for .NET Copyright (C) Microsoft Corporation. All rights reserved. Determining projects to restore... Restored /Users/phi/ws/helloworld/helloworld.csproj (in 216 ms). You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview helloworld -> /Users/phi/ws/helloworld/bin/Debug/net6.0/osx-arm64/helloworld.dll helloworld -> /Users/phi/ws/helloworld/arm64/ Non-fat file: arm64/helloworld is architecture: arm64 zsh: killed ./arm64/helloworld ``` Machine information: ``` % sw_vers ProductName: macOS ProductVersion: 11.1 BuildVersion: 20C69 % system_profiler -detailLevel mini SPHardwareDataType Hardware: Hardware Overview: Model Name: Mac mini Model Identifier: Macmini9,1 Chip: Apple M1 Total Number of Cores: 8 (4 performance and 4 efficiency) Memory: 8 GB System Firmware Version: 6723.61.3 % dotnet --version 6.0.100-preview.1.21103.13 % dotnet --list-runtimes Microsoft.AspNetCore.App 6.0.0-preview.1.21103.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App] Microsoft.NETCore.App 6.0.0-preview.1.21102.12 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] % dotnet --list-sdks 6.0.100-preview.1.21103.13 [/usr/local/share/dotnet/sdk] ```
Author: iamphi
Assignees: -
Labels: `arch-arm64`, `area-Host`, `os-mac-os-x-big-sur`, `untriaged`
Milestone: -
mangod9 commented 3 years ago

Moving to host area since the published host needs signing. Believe there is a separate issue tracking that.

vitek-karas commented 3 years ago

@VSadov do we have an issue tracking the "sign apphost in SDK on Mac" ?

VSadov commented 3 years ago

I am not sure if we have an issue for this.

It looks more of a documentation issue though. The apphost (or a singlefile executable) will have to be signed or notarized or otherwise comply with distribution requirements on MacOS. These steps typically must be done by the developer and on an actual Mac machine. i.e. you can build a Mac exe on Linux or Windows, but signing still requires a Mac, so ultimately there will be steps that need to be done by the developer.

We could probably ad-hoc sign when building on Mac, but I am not sure if that is always the right thing to do.

CC: @samsp-msft

vitek-karas commented 3 years ago

It would be really weird if dotnet run would not "just work". Today we avoid the problem by actually calling dotnet app.dll under the hood, but if we turn on automatic generation of apphost for MacOS, it should call app in which case it would just fail.

Ideally we would sign dotnet build produces builds on MacOS - for the above case to work. It might make sense to not sign dotnet publish output - partly due to the reason you mention (it would not work the same across platforms) and partly because it would probably make it cumbersome to real-sign the output.

agocke commented 3 years ago

This should be fixed now that we're dev signing during publish https://github.com/dotnet/runtime/issues/50375

parkycai commented 3 years ago

Looks like it's broken again in preview 7

mangod9 commented 3 years ago

@parkycai could you please provide details on what scenario you are noticing is broken?

parkycai commented 3 years ago

@parkycai could you please provide details on what scenario you are noticing is broken?

--self-contained is working, but with -p:PublishSingleFile=true, it's broken.

Test Script:

mkdir helloworld
cd helloworld 
dotnet new console

echo "### Testing dotnet run"
dotnet run

echo "### Testing intel publish"
dotnet publish --self-contained -r osx-x64 -p:PublishSingleFile=true -o intel
lipo -info intel/helloworld
./intel/helloworld                                                     

echo "### Testing arm64 publish"
dotnet publish --self-contained -r osx-arm64 -p:PublishSingleFile=true -o arm64
lipo -info arm64/helloworld
./arm64/helloworld

My output:

The template "Console Application" was created successfully.

Processing post-creation actions...
Running 'dotnet restore' on /Users/waynecai/Documents/Workspaces/NetCore/helloworld/helloworld.csproj...
  Determining projects to restore...
  Restored /Users/waynecai/Documents/Workspaces/NetCore/helloworld/helloworld.csproj (in 126 ms).
Restore succeeded.

### Testing dotnet run
Hello, World!
### Testing intel publish
Microsoft (R) Build Engine version 17.0.0-preview-21378-03+d592862ed for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  Restored /Users/waynecai/Documents/Workspaces/NetCore/helloworld/helloworld.csproj (in 161 ms).
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  helloworld -> /Users/waynecai/Documents/Workspaces/NetCore/helloworld/bin/Debug/net6.0/osx-x64/helloworld.dll
  helloworld -> /Users/waynecai/Documents/Workspaces/NetCore/helloworld/intel/
Non-fat file: intel/helloworld is architecture: x86_64
Hello, World!
### Testing arm64 publish
Microsoft (R) Build Engine version 17.0.0-preview-21378-03+d592862ed for .NET
Copyright (C) Microsoft Corporation. All rights reserved.

  Determining projects to restore...
  Restored /Users/waynecai/Documents/Workspaces/NetCore/helloworld/helloworld.csproj (in 159 ms).
  You are using a preview version of .NET. See: https://aka.ms/dotnet-core-preview
  helloworld -> /Users/waynecai/Documents/Workspaces/NetCore/helloworld/bin/Debug/net6.0/osx-arm64/helloworld.dll
  helloworld -> /Users/waynecai/Documents/Workspaces/NetCore/helloworld/arm64/
Non-fat file: arm64/helloworld is architecture: arm64
zsh: killed     ./arm64/helloworld

I notice that codesign -s - ./arm64/helloworld reports ./arm64/helloworld: is already signed

and codesign -f -s - ./arm64/helloworld reports 

./arm64/helloworld: replacing existing signature
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/codesign_allocate: object: /Users/waynecai/Documents/Workspaces/NetCore/helloworld/arm64/helloworld malformed object (code signature data at offset 9994656 with a size of 96384, overlaps string table at offset 8608472 with a size of 62057869)
./arm64/helloworld: the codesign_allocate helper tool cannot be found or used

Enviroments:

ProductName:    macOS
ProductVersion: 11.5
BuildVersion:   20G71
Hardware:

    Hardware Overview:

      Model Name: MacBook Air
      Model Identifier: MacBookAir10,1
      Chip: Apple M1
      Total Number of Cores: 8 (4 performance and 4 efficiency)
      Memory: 16 GB
      System Firmware Version: 6723.140.2
      OS Loader Version: 6723.140.2

6.0.100-preview.7.21379.14
Microsoft.AspNetCore.App 6.0.0-preview.7.21378.6 [/usr/local/share/dotnet/shared/Microsoft.AspNetCore.App]
Microsoft.NETCore.App 6.0.0-preview.7.21377.19 [/usr/local/share/dotnet/shared/Microsoft.NETCore.App]
6.0.100-preview.7.21379.14 [/usr/local/share/dotnet/sdk]
parkycai commented 3 years ago

I'm opening a new issue.