ghost
commented
2 years ago Tagging subscribers to this area: @tommcdon See info in area-owners.md if you want to be subscribed.
Issue Details
### Description We're a MSFT internal team using dotnet on linux in AKS. We're onboarded to Azure Watson and some of our teams are looking to migrate to distroless containers for our applications. We have noticed that collecting core dumps on crashes does not work correctly unless we start dotnet inside a shell script, aka this doesn't generate the correct core dumps: ```dockerfile ENTRYPOINT ["dotnet", "MyApp.dll"] ``` however this does: ```dockerfile ENTRYPOINT ["/bin/bash", "start.sh"] ``` where `start.sh` is a small wrapper that forwards signals and calls `dotnet` itself. We're expecting 2 core dumps to be taken: - One by dotnet using createdump - One by the kernel itself However, if we don't run with a shell, we typically see 0 dumps when the app crashes (we do occasionally see 1 dump, which is weird). ### Reproduction Steps The app code seems to be irrelevant, anything that crashes the process seems to work: ```csharp namespace MyApplication { using System; using System.Threading; public class Program { public static void Main(string[] args) { Thread.Sleep(5000); throw new Exception("Crashed on startup"); } } } ``` Then a dockerfile like this one: ```dockerfile FROM mcr.microsoft.com/dotnet/runtime:6.0-bullseye-slim ARG source=./ WORKDIR /app # This is just the result of `dotnet publish` COPY $source . ENTRYPOINT ["dotnet", "MyApplication.dll"] ``` We then deploy it to AKS in a pod with `COMPlus_DbgEnableMiniDump=1` and `COMPlus_DbgMiniDumpType=4`, and the azure-watson agent running on the node. ### Expected behavior createdump takes a dump of the managed heap and the watson agent finds it and uploads it to the portal. ### Actual behavior Generally, no dump appears. Occassionally, a dump will show up without symbols. One thing that is a little notable is that these dumps typically are in a SIGSEGV bucket, whereas dumps that come from containers running from a shell are almost always in a SIGABRT bucket. ### Regression? n/a ### Known Workarounds As mentioned above. The issues only comes up if we are running dotnet as the entrypoint in our container. If the entrypoint is a bash script that starts dotnet. Everything works as expected, and our core dumps are properly taken and uploaded. We haven't tried other shells or init exe's ### Configuration We're using `mcr.microsoft.com/dotnet/runtime:6.0-bullseye-slim` and I believe we've seen this with the `aspnet` version as well. Host OS : `Ubuntu 18.04.6 LTS` Kernel version: `5.4.0-1078-azure` Arch: `x64` ### Other information _No response_
| Author: | peter-glotfelty |
|---|---|
| Assignees: | - |
| Labels: | `area-Diagnostics-coreclr`, `untriaged` |
| Milestone: | - |
hoyosjs
peter-glotfelty
mikem8361
tommcdon
Description
We're a MSFT internal team using dotnet on linux in AKS. We're onboarded to Azure Watson and some of our teams are looking to migrate to distroless containers for our applications. We have noticed that collecting core dumps on crashes does not work correctly unless we start dotnet inside a shell script, aka this doesn't generate the correct core dumps:
however this does:
where
start.shis a small wrapper that forwards signals and callsdotnetitself.We're expecting 2 core dumps to be taken:
However, if we don't run with a shell, we typically see 0 dumps when the app crashes (we do occasionally see 1 dump, which is weird).
Reproduction Steps
The app code seems to be irrelevant, anything that crashes the process seems to work:
Then a dockerfile like this one:
We then deploy it to AKS in a pod with
COMPlus_DbgEnableMiniDump=1andCOMPlus_DbgMiniDumpType=4, and the azure-watson agent running on the node.Expected behavior
createdump takes a dump of the managed heap and the watson agent finds it and uploads it to the portal.
Actual behavior
Generally, no dump appears. Occassionally, a dump will show up without symbols. One thing that is a little notable is that these dumps typically are in a SIGSEGV bucket, whereas dumps that come from containers running from a shell are almost always in a SIGABRT bucket.
Regression?
n/a
Known Workarounds
As mentioned above. The issues only comes up if we are running dotnet as the entrypoint in our container. If the entrypoint is a bash script that starts dotnet. Everything works as expected, and our core dumps are properly taken and uploaded. We haven't tried other shells or init exe's
Configuration
We're using
mcr.microsoft.com/dotnet/runtime:6.0-bullseye-slimand I believe we've seen this with theaspnetversion as well.Host OS :
Ubuntu 18.04.6 LTSKernel version:
5.4.0-1078-azureArch:x64Other information
No response