Closed sdmaclea closed 2 years ago
We're intending on releasing Apple M1 support (native and R2) starting with .NET 6.0 Preview 1 (first half of February). There will almost certainly be issues and feature gaps in Preview 1. Support will get better with each preview. For example, Preview 1 will have only native SOS-style debugging support and we intend to add managed Visual Studio-style debugging with Preview 3.
Our current thinking is to backport R2 support to .NET 5.0 and .NET Core 3.1, and that native M1 support will be .NET 6.0 only. The change to support native is quite significant, and we don't see a good cost/risk/benefit tradeoff.
I was busy with meetings yesterday, so apologies for the delay in replies. There are a few important fixes in the latest 11.2 beta, and I'm glad that it is working well for people in practice.
@richlander What .NET changes are you thinking of backporting for Rosetta 2 support? I don't think any should be required with 11.2.
@zwarich -- good question. I'm not up to date on 11.2 beta. I'll pass to the question to @janvorli. Do we need to make any changes to .NET x64 to enable correct and optimal operation in R2?
@richlander Since Apple made all the changes in Rosetta 2, we think this will just work for .NET 2.1, .NET 3.1, and .NET 5 w/o any changes to the runtime/SDK.
Agree with @sdmaclea, I believe we won't need to make any changes to make Rosetta work.
There is one last issue that we hit when running all the .NET Core pri1 tests under Rosetta 2 - the assertion failed: GPR thread_set_state is unsupported while in sa_tramp. (ThreadContextRegisterState.cpp:1250 thread_set_state_gpr_64)
listed in the head of this github issue. We have seen that failing in the past, as can be seen in the discussion above. I am currently investigating it to get more details so that I can report it to Apple.
@janvorli That issue will be fixed in a future release. I didn't see it reproduce when running the pri1 tests myself, but it makes sense that some of the tests could hit it. Please file a radar if there's an individual test that can reproduce it fairly consistently when run in a loop, but otherwise your time is probably better spent elsewhere.
Thanks @zwarich, I will file it. It reproduces consistently and in 100% of the test launches for me in two of the tests (that were mentioned above) when running the tests in unchanged .NET runtime master branch. Here are the repro steps:
Edit: I've fixed the -coreroot option above, it was missing the =
@zwarich I've filed it in as FB8970566
Not sure if this is going to help, but after extensive use of macOS 11.2 Beta 2 with me and my team, we discovered a sudden runtime failure.
We've ran the project multiple times and it only happens once every 20-30 runs.
info: System.Net.Http.HttpClient.Default.ClientHandler[100]
Sending HTTP request GET https://localhost:6001/ApiKey/All
info: System.Net.Http.HttpClient.Default.ClientHandler[101]
Received HTTP response headers after 365.6311ms - 200
info: System.Net.Http.HttpClient.Default.LogicalHandler[101]
End processing HTTP request after 371.5579ms - 200
assertion failed [code_fragment->kind == CodeFragmentKind::InvalidJit]: metadata kind is not CodeFragmentKind::InvalidJit
(TranslationCacheJit.cpp:262 handle_jit_breakpoint)
@richlander this means about another year of slowness of .net for m1 when competing platforms like Java have already had native support for weeks. What is the issue of trying to move native in 5.0. Can someone try to apply the changes to 5.0 and see how far off it is for native ?
Kind regards, Ilya Beyrak | Founder & CEO Resultly LLC. (p) 312.273.9401 (m) 847.912.1212 (w) www.result.ly ( http://www.result.ly/ )
Sent via Superhuman iOS ( https://sprh.mn/?vip=ilya@resultly.com )
On Thu, Jan 14 2021 at 10:54 AM, Rich Lander < notifications@github.com > wrote:
@zwarich ( https://github.com/zwarich ) -- good question. I'm not up to date on 11.2 beta. I'll pass to the question to @janvorli ( https://github.com/janvorli ). Do we need to make any changes to .NET x64 to enable correct and optimal operation in R2?
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub ( https://github.com/dotnet/runtime/issues/44897#issuecomment-760323940 ) , or unsubscribe ( https://github.com/notifications/unsubscribe-auth/AASG62LBT6L6ULTL7KL6K7LSZ4OVJANCNFSM4T2RPPWQ ).
@ilushka85 100% this! I'm kind of stunned by this decision and still in disbelieve that there won't be official native support by MS for a long time while the rest of the world has either already completed the transition or is close to doing so. Just to make this very clear: You're not hurting Apple or it's platforms with your decision but your own developers and users of their apps. Users don't care which development framework an app uses, they care very much about an app being native and fast though. So I sincerely hope that you can escape your bubble ASAP and realize that reality has moved on since you last checked.
For what it is worth, as a Windows on ARM user as well ( Surface Pro X ), even there, .NET support is relatively weak. VS is not native. There is little to no .NET ARM64 debugger support on Windows on ARM. So while it is an annoying situation, I think we need to maintain a bit of perspective. As of right now, it looks like the Apple Silicon support will arrive before fully Windows on ARM support really does. ( only sort of tongue in cheek )
As the owner of a new M1 MacBook Air this is pretty disappointing. I do my day to day work on a desktop PC, but my laptop for everything else.
I've recommended .NET Core to friends who run a startup and been running happily on Intel Macs (no PCs at all in this shop) for 2+ years with no problems. Fast forward to today, with most of their MacBooks exhibiting the butterfly keyboard problems, they upgraded to M1 MacBook Pros. I should have just recommended they used Elixir. Come on Microsoft - this can't wait till November/6.0.
@nixxholas I haven't seen any reports of problems like that before. Is it possible for you to file an issue with Feedback Assistant that includes all of the the binaries required to reproduce that issue? If so, it would be greatly appreciated and would accelerate a fix.
For what it is worth, as a Windows on ARM user as well ( Surface Pro X ), even there, .NET support is relatively weak. VS is not native. There is little to no .NET ARM64 debugger support on Windows on ARM. So while it is an annoying situation, I think we need to maintain a bit of perspective. As of right now, it looks like the Apple Silicon support will arrive before fully Windows on ARM support really does. ( only sort of tongue in cheek )
I dont think its acceptable that there is no support on Windows on Arm for the debugger. But visual studio has proven to be junk in recent years while staying windows only and 32 bit... This is where rider truly shines. But the fact that the .net platform itself is going to take a year to even consider launching on native apple silicon is troubling when every other platform has done it.
The fact that the m1 support is there in 6.0 and its just a decision of we dont want to back port to 5.0... considering the changes are not even that great.
I understand that people want Apple M1 support now/yesterday. We're working as quickly as we can. I'm hoping that the experience is fully functional and stable by about .NET 6.0 Preview 4. Some of the Apple M1 chip characteristics have required us to make significant changes, above and beyond our existing quite functional Arm64 support. It's possible that other development platforms had fewer changes to make. We'll likely do a blog post on this to help contextualize our project.
Some of the comments suggested company politics are the cause of the delay. That is absolutely untrue. On the .NET Team, we have a great relationship with Apple and very much want .NET running natively on M1 chips ASAP. We have been working directly with Apple engineers to that end. They are very talented and have offered us significant help. I can assure you the only Microsoft company direction we are getting is to "go faster".
Windows Arm64 -- also raised in the comments -- is another similar project. The two projects have different needs, and so are largely not bottlenecked on the same people. The only place were we do have some (human) resource contention is the Visual Studio debugger. We're working on that.
The first big description of where we are at will be 6.0 Preview 1. That's targeted for somewhere around 2/9.
@richlander We all appreciate the effort and difficulty in getting .NET Core fully supported on M1, but waiting until November is completely unrealistic for folks who need to ship code.
.NET has a big perception problem in the open source / startup community where it's seen as a framework/system for large enterprises. With .NET Core Microsoft has made huge inroads in changing that perception, but decisions like this just flat out hurt new .NET adoption.
Folks aren't going to wait around for 10 months before they can ship code on a non beta/preview release. Especially as Apple starts shipping more and more Apple Silicon machines. .NET 5.X on Apple Silicon should be a priority.
This decision gives an already reluctant community (startups, open source projects, etc.) less reason to even consider .NET Core. Go released a beta with native M1 support in December Rust is also making significant inroads to supporting Apple Silicon. Meanwhile Microsoft is going to take it's sweet time and I guess those of us with these amazing new machines are supposed to just wait around for 10 months. No, we don't want to be your 6.0 alpha/beta testers.
@richlander why not attempt to prioritize even a beta 5.0 version with support for m1? I think @GiorgioG comments highlight some of the recent issues with inroads and backwardness in .net with the move to 5.0 and even azure alignment.
.Net 5 was released in December and yet there is still no support in Azure Functions for it. You would think that the alignment of .net / functions / Microsoft / azure would mean these things are coordinated together but the fact that we are here months after a release with no solution for azure functions is troubling.
Now the position of we won't try to get native support out til the next framework version a year away is just crazy.... When will azure functions support 6.0 by the way?
Realistically speaking what would it take to attempt to move the changes for m1 to the 5.0 branch? What can we do to help ?
What can we do to help?
Build your apps with the master branch and ship them as self-contained? Btw why do you need native support if apple's emulator is great?
What can we do to help?
Build your apps with the master branch and ship them as self-contained? Btw why do you need native support if apple's emulator is great?
Is this a real response? Most of us are developers and trying to use these machines for development. Apples great emulator until beta 2 a couple of days ago didn't even work with .net for starters. Second of all while rosetta two works "great" its still not anywhere near native speed especially for jit apps like .net / java. The performance is staggering and makes working as a developer in .net on these machines unnecessarily hard.
And your suggestion of even using the latest nightlies of .net 6 is flawed... have you tried .net 6? Do you think it works well ? I want to spend my time developing and debugging my app not figuring out why .net 6 doesnt work(it shouldn't its not done)
Btw why do you need native support if apple's emulator is great?
I'd be happy if .NET Core worked in any fashion in the short term. But I do need .NET Core to work for development on M1 machines.
@richlander We all appreciate the effort and difficulty in getting .NET Core fully supported on M1, but waiting until November is completely unrealistic for folks who need to ship code
Let's test an assumption I have. My assumption is that the vast majority of people using .NET on macOS are building web sites, as opposed to client apps. If folks are building web sites, then .NET 5.0 as-is seems perfectly satisfactory. I agree that it is not awesome, but it hasn't made .NET worse. In fact, if you believe Apple's marketing, it will run better than on x64 machines. If folks have performance metrics that demonstrate emulated .NET running worse on M1 machines, that would be super helpful.
For folks building client apps, that's different. I'd like to learn more about people using .NET (not Xamarin) to build macOS client apps. That information would help.
@richlander @GiorgioG Just my opinion, .NET 5 (and even Core 3.1) on Rosetta seems to be an acceptable mitigation in the wait for native support.
About those metrics, please provide some test scenario's/instructions and I'm happy to benchmark between a MBP16 i7 and an MBA M1
FWIW, I was able to debug my asp.net core 3.1 app on M1, but the performance seems to be about 10% as fast as a native x86 system. This is purely from a development perspective and not release performance.
Some observations:
If I can help/test anything just let me know.
UPDATE: migrating to .net 5.0 demonstrates the same behavior.
About those metrics, please provide some test scenario's/instructions
Thanks for the offer. That said, I'm more interested in people's real-world experiences as opposed to synthetic benchmarks.
@mcoolidge -- thanks for the report. We'll see if we can repro that. Do you mean that you are running an ASP.NET Core service, and that when you call it (say with curl) that it is very slow or unresponsive?
I am debugging an asp.net core web api + SPA. The services called from the JS side are much slower and I can see some of the dotnet debug logging running much slower. It's not really practical to use for development. Also, the SPA proxy stops working sporadically but works after restarting.
For now, my workaround is to use remote debug with a windows box and accessing the page on the mac.
If needed, I can do some profiling on the m1 (if someone can point me in the right direction).
Thanks @mcoolidge. That's enough for now. I'll talk with folks tomorrow (today is a holiday).
@richlander, like @mcoolidge I'm also working on an asp.net core web api project + SPA. Being unable to debug is a showstopper.
@GiorgioG what does it mean "being unable to debug" in your case? Do you experience similar issues as @mcoolidge or some other problems? And are you running on the macOS 11.2 beta 2?
@janvorli - Apologies for the late response - I was running macOS 11.1. Upgrading to the 11.2 beta appears to have resolved the issue of being unable to debug (in VS for Mac as well as JetBrains Rider.)
@GiorgioG thank you for confirming that it works for you with the latest macOS beta.
A quick update: Running the new macOS Beta does seem to help with crashes, but performance is still pretty bad. Even when using dotnet run, the API responses are rather sluggish and I often see:
Heartbeat
took longer than "00:00:01" at "01/27/2021 16:06:30 +00:00". This could be caused by thread pool starvation.`
@mcoolidge if you run your application without a debugger, do you get the same performance issues? We've had some problems with app being sluggish on x64 mac when running under a debugger in the past and I wonder if this could be related.
I see the same from doing debug or just "dotnet run".
Do you mean using a different configuration? I don't think running the command attaches a debugger, but I might be wrong.
No, I meant just "dotnet run".
@janvorli - my observation is that dotnet applications have good performance on M1 without the debugger attached, but they're quite slow with the debugger attached.
Im often randomly getting:
this is with latest OSX 11.2 update. .net 5
assertion failed [code_fragment->kind == CodeFragmentKind::InvalidJit]: metadata kind is not CodeFragmentKind::InvalidJit (TranslationCacheJit.cpp:262 handle_jit_breakpoint)
I've come up with a workaround for the Apple Silicon M1 debugging problems on BigSur 11.1 and using dotnet 3.1. For work I can't use the 11.2 beta on my M1.
I build and run my dotnet code in Docker (Apple Silicon version) using an ARM64 dotnet 3.1 SDK and installed vsdgb separately in the container base image.
I can then use VSCode (insiders) to attach to the running docker container and debug the code. So far breakpoints, stepping through code and stepping into code are working !! Happy Days 👍
I'm on Big Sur 11.2 RC3 20D64 Nothing shocking here, it just works, performance is similar to the previous builds of macOS Big Sur 11.2
I must say that, compared to my Angular workflow, .NET is slow (mostly due to Rosetta).
My workflow contains of an Angular FE, .NET Core 3.1 BE with an API, WEB and medium sized solution (about 15 projects).
Have not encountered any issues yet, except the performance
@janvorli - my observation is that dotnet applications have good performance on M1 without the debugger attached, but they're quite slow with the debugger attached.
Same here, is there a ticket for this somewhere?
Im often randomly getting:
this is with latest OSX 11.2 update. .net 5
assertion failed [code_fragment->kind == CodeFragmentKind::InvalidJit]: metadata kind is not CodeFragmentKind::InvalidJit (TranslationCacheJit.cpp:262 handle_jit_breakpoint)
@sdmaclea is this a known issue?
@sdmaclea I have a feeling this might be happening sometimes when there are exceptions... but difficult to tell because it terminates everything.
I think this because I had a coding error which caused a NRE everytime... and I was not seeing any exceptions but the invalidjit error.
I ran on intel processor and saw it was reliably causing an exception.
Although I have had other exceptions work without issue.... I think! (nothing is for certain)
Other information... happens under debugger using jetbrains rider.
@danwalmsley Apple has been driving the Rosetta 2 fixes. It would help if you provide feedback to them via the developer feedback tool. That helps them track issues people are seeing in the wild and prioritize them.
/cc @zwarich
@sdmaclea shall do :) thanks
@danwalmsley if you have already reported the issue via Apple feedback tool, could you please share the feedback number?
@janvorli FB8989324
@danwalmsley thank you!
I installed the .tar.gz
option for 5.0.3 @ https://dotnet.microsoft.com/download/dotnet/5.0. Is that build unsigned? It is behaving as if it were (tons of malicious software warnings). If I use the .pkg
option, I get expected behavior. I see the same behavior with the nightly 6.0 P1 build. I would like to use the .tar.gz
option so that I can play with any .NET version privately (no global install).
I cannot get .NET 5 ASP.NET amd64 container images to work on M1. It is apparently supposed to work.
An Arm64 image works:
rich@MacBook-Air ~ % docker run --rm -p 8080:80 mcr.microsoft.com/dotnet/samples:aspnetapp
A small sample works with amd64:
docker run --platform linux/amd64 --rm -p 8080:80 mcr.microsoft.com/dotnet/samples
My experience with the aspnetapp
sample as amd64:
rich@MacBook-Air ~ % docker run --platform linux/amd64 --rm -p 8080:80 mcr.microsoft.com/dotnet/samples:aspnetapp
Unhandled exception. System.IO.IOException: Function not implemented
at System.IO.FileSystemWatcher.StartRaisingEvents()
at System.IO.FileSystemWatcher.StartRaisingEventsIfNotDisposed()
at System.IO.FileSystemWatcher.set_EnableRaisingEvents(Boolean value)
at Microsoft.Extensions.FileProviders.Physical.PhysicalFilesWatcher.TryEnableFileSystemWatcher()
at Microsoft.Extensions.FileProviders.Physical.PhysicalFilesWatcher.CreateFileChangeToken(String filter)
at Microsoft.Extensions.FileProviders.PhysicalFileProvider.Watch(String filter)
at Microsoft.Extensions.Configuration.FileConfigurationProvider.<.ctor>b__1_0()
at Microsoft.Extensions.Primitives.ChangeToken.OnChange(Func`1 changeTokenProducer, Action changeTokenConsumer)
at Microsoft.Extensions.Configuration.FileConfigurationProvider..ctor(FileConfigurationSource source)
at Microsoft.Extensions.Configuration.Json.JsonConfigurationSource.Build(IConfigurationBuilder builder)
at Microsoft.Extensions.Configuration.ConfigurationBuilder.Build()
at Microsoft.Extensions.Hosting.HostBuilder.BuildAppConfiguration()
at Microsoft.Extensions.Hosting.HostBuilder.Build()
at aspnetapp.Program.Main(String[] args) in /source/aspnetapp/Program.cs:line 16
qemu: uncaught target signal 6 (Aborted) - core dumped
Apple has announced plans to transition its Mac hardware line to a new Arm64-based chip that they refer to as “Apple Silicon”.
Initial .NET support will be through .NET running on the Rosetta 2 emulator. Longer term native support for Apple Silicon is planned for .NET 6.
While it is hoped that Rosetta 2 emulation will just work, the .NET runtime is complicated and real issues will make this a non-trivial task.
Current known issues
[x] Apple Silicon uses a 16K memory page size. The .NET 5 stack probe code doesn't handle this yet. #45226. Per Apple this only affects the DTK and is fixed on M1 Silicon. Edit: I've verified that on real M1 device, the page size is 4K and the related issue doesn't occur.
[x] Rosetta 2 emulation crashes with a fatal failure when calling with
thread_get_state
x86_FLOAT_STATE64
. This is because the emulator does not emulate AVX support, but the function should simply return an error. Edit: This is fixed in the macOS 11.2 beta release.[x] Rosetta 2 emulation doesn't populate
exceptionState.__trapno
for other kernel entry than hardware exceptions (for example for syscalls). This means we fail to inject code necessary for garbage collection and sometimes deadlock. Edit: This is at least partially fixed in the macOS 11.2 beta release, but I can still see hangs during .NET runtime / tests managed parts compilation. It is being investigated at the Apple side.[ ]
With #45226 & https://github.com/janvorli/runtime/commit/aee81acd99b9c0e6a406bad3b98c278669c7cc67 19 runtime tests are failing under Rosetta 2 emulation which pass on macOS native x64All the coreclr Pri 1 tests are now passing except two tests (mentioned in the comments below) that are failing with:assertion failed: GPR thread_set_state is unsupported while in sa_tramp. (ThreadContextRegisterState.cpp:1250 thread_set_state_gpr_64)
[x] Debugging using VS Code doesn't work. It was partially fixed in the macOS 11.2 beta release, now it is possible to successfully run an application under the debugger and break on a breakpoint. But attempt to single step or continue from that state still fails. It is caused by iret instruction emulation that doesn't honor the trace flag. Edit: This is fixed in macOS 11.2 beta 2
[x] New issue in macOS 11.2 beta - dotnet build and posibly other .NET applications often fail with
assertion failed [abi_info.u.translated_code.instruction_extents.kind == InstructionOffsetKind::Syscall]: on sigreturn exit path but instruction isn't marked as a syscall (ThreadContextRegisterState.cpp:381 x86_gpr_state_from_arm_state)
Edit: This is fixed in macOS 11.2 beta 2