search

dotnet / diagnostics

This repository contains the source code for various .NET Core runtime diagnostic tools and documents.
MIT License
1.18k stars 355 forks source link

Necessary permissions in docker using a rootless user for dotnet-* tools #2102

Open DOMZE opened 3 years ago

DOMZE commented 3 years ago

Hello,

I'm using the dotnet-dump / dotnet-trace within a docker container, engine running in WSL2, using a rootless user.

dotnet-dump In my docker file, I added CAP_SYS_PTRACE capabilities to createdump using setcap CAP_SYS_PTRACE=+eip $(find /usr/share -name createdump) I also start my container with --cap-add=SYS_PTRACE

Once I did that I was able to get passed the error Core dump generation FAILED 0x80004005 due to PTrace(ATTACH, 1) FAILED Operation not permitted

However now I'm getting the error /usr/share/dotnet/shared/Microsoft.NETCore.App/5.0.4/createdump: error while loading shared libraries: libmscordaccore.so: cannot open shared object file: No such file or directory

chowing recursively the directory /usr/share/dotnet/shared/Microsoft.NETCore.App to a group where my user is in also has no impact.

is root absolutely necessary to create dumps? If this is the case, is running a side-car container my only option to be able to generate dumps if my main application container runs under a rootless user?

dotnet-trace Using the same rootless user, if I try to create a trace, I get the following error:

No profile or providers specified, defaulting to trace profile 'cpu-sampling'

Provider Name                           Keywords            Level               Enabled By
Microsoft-DotNETCore-SampleProfiler     0x0000F00000000000  Informational(4)    --profile
Microsoft-Windows-DotNETRuntime         0x00000014C14FCCBD  Informational(4)    --profile

[ERROR] System.NullReferenceException: Object reference not set to an instance of an object.
   at Microsoft.Diagnostics.Tools.Trace.CollectCommandHandler.Collect(CancellationToken ct, IConsole console, Int32 processId, FileInfo output, UInt32 buffersize, String providers, String profile, TraceFileFormat format, TimeSpan duration, String clrevents, String clreventlevel, String name, String diagnosticPort) in /_/src/Tools/dotnet-trace/CommandLine/Commands/CollectCommand.cs:line 163

what permissions is necessary to capture tracing using a rootless user?

Thank you!

mikem8361 commented 3 years ago

dotnet-dump should only need the SYS_PTRACE capability (--cap-add=SYS_PTRACE should be enough). I'm not sure what your setcap command does exactly. Can you send what ls -l (should display the user of each file) in /usr/share/dotnet/shared/Microsoft.NETCore.App/5.0.4/? For some reason, the loader can't find libmscordaccore.so. It is a shared module reference by createdump.

DOMZE commented 3 years ago

The setcap command with CAP_SYS_PTRACE adds the capability for a process to trace arbitrary processes using ptrace. If i don't setcap, I automatically get Core dump generation FAILED 0x80004005 due to PTrace(ATTACH, 1) FAILED Operation not permitted

Running either of those 2 commands fail with 

/usr/share/dotnet/shared/Microsoft.NETCore.App/5.0.4/createdump: error while loading shared libraries: libmscordaccore.so: cannot open shared object file: No such file or directory

chown -R root:testuser /usr/share/dotnet/shared chown -R testuser:testuser /usr/share/dotnet/shared

The output below is the output of the last command

total 73652
-rw-r--r-- 1 testuser testuser  785408 Feb 16 22:19 Microsoft.CSharp.dll
-rw-r--r-- 1 testuser testuser  120997 Feb 16 22:20 Microsoft.NETCore.App.deps.json
-rw-r--r-- 1 testuser testuser 1172992 Feb 16 22:19 Microsoft.VisualBasic.Core.dll
-rw-r--r-- 1 testuser testuser    8704 Feb 16 22:19 Microsoft.VisualBasic.dll
-rw-r--r-- 1 testuser testuser   12800 Feb 16 22:19 Microsoft.Win32.Primitives.dll
-rw-r--r-- 1 testuser testuser   51712 Feb 16 22:19 Microsoft.Win32.Registry.dll
-rw-r--r-- 1 testuser testuser    5632 Feb 16 22:19 System.AppContext.dll
-rw-r--r-- 1 testuser testuser    5632 Feb 16 22:19 System.Buffers.dll
-rw-r--r-- 1 testuser testuser  184832 Feb 16 22:19 System.Collections.Concurrent.dll
-rw-r--r-- 1 testuser testuser  666624 Feb 16 22:19 System.Collections.Immutable.dll
-rw-r--r-- 1 testuser testuser   91136 Feb 16 22:19 System.Collections.NonGeneric.dll
-rw-r--r-- 1 testuser testuser   85504 Feb 16 22:19 System.Collections.Specialized.dll
-rw-r--r-- 1 testuser testuser  279040 Feb 16 22:19 System.Collections.dll
-rw-r--r-- 1 testuser testuser  168448 Feb 16 22:19 System.ComponentModel.Annotations.dll
-rw-r--r-- 1 testuser testuser    8192 Feb 16 22:19 System.ComponentModel.DataAnnotations.dll
-rw-r--r-- 1 testuser testuser   28672 Feb 16 22:19 System.ComponentModel.EventBasedAsync.dll
-rw-r--r-- 1 testuser testuser   54272 Feb 16 22:19 System.ComponentModel.Primitives.dll
-rw-r--r-- 1 testuser testuser  698368 Feb 16 22:19 System.ComponentModel.TypeConverter.dll
-rw-r--r-- 1 testuser testuser    8704 Feb 16 22:19 System.ComponentModel.dll
-rw-r--r-- 1 testuser testuser   10752 Feb 16 22:19 System.Configuration.dll
-rw-r--r-- 1 testuser testuser  180224 Feb 16 22:19 System.Console.dll
-rw-r--r-- 1 testuser testuser   15360 Feb 16 22:19 System.Core.dll
-rw-r--r-- 1 testuser testuser 2908672 Feb 16 22:19 System.Data.Common.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Data.DataSetExtensions.dll
-rw-r--r-- 1 testuser testuser   17408 Feb 16 22:19 System.Data.dll
-rw-r--r-- 1 testuser testuser    6656 Feb 16 22:19 System.Diagnostics.Contracts.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Diagnostics.Debug.dll
-rw-r--r-- 1 testuser testuser  176640 Feb 16 22:19 System.Diagnostics.DiagnosticSource.dll
-rw-r--r-- 1 testuser testuser   29696 Feb 16 22:19 System.Diagnostics.FileVersionInfo.dll
-rw-r--r-- 1 testuser testuser  240640 Feb 16 22:19 System.Diagnostics.Process.dll
-rw-r--r-- 1 testuser testuser   26112 Feb 16 22:19 System.Diagnostics.StackTrace.dll
-rw-r--r-- 1 testuser testuser   50176 Feb 16 22:19 System.Diagnostics.TextWriterTraceListener.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Diagnostics.Tools.dll
-rw-r--r-- 1 testuser testuser  117760 Feb 16 22:19 System.Diagnostics.TraceSource.dll
-rw-r--r-- 1 testuser testuser    7168 Feb 16 22:19 System.Diagnostics.Tracing.dll
-rw-r--r-- 1 testuser testuser  116736 Feb 16 22:19 System.Drawing.Primitives.dll
-rw-r--r-- 1 testuser testuser   12288 Feb 16 22:19 System.Drawing.dll
-rw-r--r-- 1 testuser testuser    6656 Feb 16 22:19 System.Dynamic.Runtime.dll
-rw-r--r-- 1 testuser testuser  180224 Feb 16 22:19 System.Formats.Asn1.dll
-rw-r--r-- 1 testuser testuser    6656 Feb 16 22:19 System.Globalization.Calendars.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Globalization.Extensions.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Globalization.dll
-rw-r--r-- 1 testuser testuser   62976 Feb 16 22:19 System.IO.Compression.Brotli.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.IO.Compression.FileSystem.dll
-rw-r--r-- 1 testuser testuser   28160 Feb 16 22:19 System.IO.Compression.ZipFile.dll
-rw-r--r-- 1 testuser testuser  239616 Feb 16 22:19 System.IO.Compression.dll
-rw-r--r-- 1 testuser testuser   24576 Feb 16 22:19 System.IO.FileSystem.AccessControl.dll
-rw-r--r-- 1 testuser testuser   66560 Feb 16 22:19 System.IO.FileSystem.DriveInfo.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.IO.FileSystem.Primitives.dll
-rw-r--r-- 1 testuser testuser   84992 Feb 16 22:19 System.IO.FileSystem.Watcher.dll
-rw-r--r-- 1 testuser testuser  205312 Feb 16 22:19 System.IO.FileSystem.dll
-rw-r--r-- 1 testuser testuser   70656 Feb 16 22:19 System.IO.IsolatedStorage.dll
-rw-r--r-- 1 testuser testuser   61440 Feb 16 22:19 System.IO.MemoryMappedFiles.dll
-rw-r--r-- 1 testuser testuser   16896 Feb 16 22:19 System.IO.Pipes.AccessControl.dll
-rw-r--r-- 1 testuser testuser  112128 Feb 16 22:19 System.IO.Pipes.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.IO.UnmanagedMemoryStream.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.IO.dll
-rw-r--r-- 1 testuser testuser 5336064 Feb 16 22:19 System.Linq.Expressions.dll
-rw-r--r-- 1 testuser testuser 1291776 Feb 16 22:19 System.Linq.Parallel.dll
-rw-r--r-- 1 testuser testuser  183296 Feb 16 22:19 System.Linq.Queryable.dll
-rw-r--r-- 1 testuser testuser  414720 Feb 16 22:19 System.Linq.dll
-rw-r--r-- 1 testuser testuser  204800 Feb 16 22:19 System.Memory.dll
-rw-r--r-- 1 testuser testuser   43520 Feb 16 22:19 System.Net.Http.Json.dll
-rw-r--r-- 1 testuser testuser 1767936 Feb 16 22:19 System.Net.Http.dll
-rw-r--r-- 1 testuser testuser  295936 Feb 16 22:19 System.Net.HttpListener.dll
-rw-r--r-- 1 testuser testuser  482304 Feb 16 22:19 System.Net.Mail.dll
-rw-r--r-- 1 testuser testuser   64000 Feb 16 22:19 System.Net.NameResolution.dll
-rw-r--r-- 1 testuser testuser  152064 Feb 16 22:19 System.Net.NetworkInformation.dll
-rw-r--r-- 1 testuser testuser   83968 Feb 16 22:19 System.Net.Ping.dll
-rw-r--r-- 1 testuser testuser  213504 Feb 16 22:19 System.Net.Primitives.dll
-rw-r--r-- 1 testuser testuser  336896 Feb 16 22:19 System.Net.Requests.dll
-rw-r--r-- 1 testuser testuser  732672 Feb 16 22:19 System.Net.Security.dll
-rw-r--r-- 1 testuser testuser   26112 Feb 16 22:19 System.Net.ServicePoint.dll
-rw-r--r-- 1 testuser testuser  588288 Feb 16 22:19 System.Net.Sockets.dll
-rw-r--r-- 1 testuser testuser  153600 Feb 16 22:19 System.Net.WebClient.dll
-rw-r--r-- 1 testuser testuser   51712 Feb 16 22:19 System.Net.WebHeaderCollection.dll
-rw-r--r-- 1 testuser testuser   18944 Feb 16 22:19 System.Net.WebProxy.dll
-rw-r--r-- 1 testuser testuser   54272 Feb 16 22:19 System.Net.WebSockets.Client.dll
-rw-r--r-- 1 testuser testuser  147968 Feb 16 22:19 System.Net.WebSockets.dll
-rw-r--r-- 1 testuser testuser    8192 Feb 16 22:19 System.Net.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Numerics.Vectors.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Numerics.dll
-rw-r--r-- 1 testuser testuser   82432 Feb 16 22:19 System.ObjectModel.dll
-rw-r--r-- 1 testuser testuser 9273344 Feb 16 21:56 System.Private.CoreLib.dll
-rw-r--r-- 1 testuser testuser 2072576 Feb 16 22:19 System.Private.DataContractSerialization.dll
-rw-r--r-- 1 testuser testuser  232960 Feb 16 22:19 System.Private.Uri.dll
-rw-r--r-- 1 testuser testuser  396288 Feb 16 22:19 System.Private.Xml.Linq.dll
-rw-r--r-- 1 testuser testuser 8548352 Feb 16 22:19 System.Private.Xml.dll
-rw-r--r-- 1 testuser testuser   62976 Feb 16 22:19 System.Reflection.DispatchProxy.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Reflection.Emit.ILGeneration.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Reflection.Emit.Lightweight.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Reflection.Emit.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Reflection.Extensions.dll
-rw-r--r-- 1 testuser testuser 1061888 Feb 16 22:19 System.Reflection.Metadata.dll
-rw-r--r-- 1 testuser testuser    6656 Feb 16 22:19 System.Reflection.Primitives.dll
-rw-r--r-- 1 testuser testuser   23552 Feb 16 22:19 System.Reflection.TypeExtensions.dll
-rw-r--r-- 1 testuser testuser    7168 Feb 16 22:19 System.Reflection.dll
-rw-r--r-- 1 testuser testuser    5632 Feb 16 22:19 System.Resources.Reader.dll
-rw-r--r-- 1 testuser testuser    6656 Feb 16 22:19 System.Resources.ResourceManager.dll
-rw-r--r-- 1 testuser testuser   35328 Feb 16 22:19 System.Resources.Writer.dll
-rw-r--r-- 1 testuser testuser    9728 Feb 16 22:19 System.Runtime.CompilerServices.Unsafe.dll
-rw-r--r-- 1 testuser testuser   10240 Feb 16 22:19 System.Runtime.CompilerServices.VisualC.dll
-rw-r--r-- 1 testuser testuser    8704 Feb 16 22:19 System.Runtime.Extensions.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Runtime.Handles.dll
-rw-r--r-- 1 testuser testuser   18432 Feb 16 22:19 System.Runtime.InteropServices.RuntimeInformation.dll
-rw-r--r-- 1 testuser testuser   39936 Feb 16 22:19 System.Runtime.InteropServices.dll
-rw-r--r-- 1 testuser testuser    7168 Feb 16 22:19 System.Runtime.Intrinsics.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Runtime.Loader.dll
-rw-r--r-- 1 testuser testuser  190976 Feb 16 22:19 System.Runtime.Numerics.dll
-rw-r--r-- 1 testuser testuser  311296 Feb 16 22:19 System.Runtime.Serialization.Formatters.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Runtime.Serialization.Json.dll
-rw-r--r-- 1 testuser testuser   18944 Feb 16 22:19 System.Runtime.Serialization.Primitives.dll
-rw-r--r-- 1 testuser testuser    7168 Feb 16 22:19 System.Runtime.Serialization.Xml.dll
-rw-r--r-- 1 testuser testuser    8192 Feb 16 22:19 System.Runtime.Serialization.dll
-rw-r--r-- 1 testuser testuser   34304 Feb 16 22:19 System.Runtime.dll
-rw-r--r-- 1 testuser testuser   61440 Feb 16 22:19 System.Security.AccessControl.dll
-rw-r--r-- 1 testuser testuser   83456 Feb 16 22:19 System.Security.Claims.dll
-rw-r--r-- 1 testuser testuser  659968 Feb 16 22:19 System.Security.Cryptography.Algorithms.dll
-rw-r--r-- 1 testuser testuser   56320 Feb 16 22:19 System.Security.Cryptography.Cng.dll
-rw-r--r-- 1 testuser testuser  107520 Feb 16 22:19 System.Security.Cryptography.Csp.dll
-rw-r--r-- 1 testuser testuser   88064 Feb 16 22:19 System.Security.Cryptography.Encoding.dll
-rw-r--r-- 1 testuser testuser  160768 Feb 16 22:19 System.Security.Cryptography.OpenSsl.dll
-rw-r--r-- 1 testuser testuser  104960 Feb 16 22:19 System.Security.Cryptography.Primitives.dll
-rw-r--r-- 1 testuser testuser  695808 Feb 16 22:19 System.Security.Cryptography.X509Certificates.dll
-rw-r--r-- 1 testuser testuser   33280 Feb 16 22:19 System.Security.Principal.Windows.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Security.Principal.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Security.SecureString.dll
-rw-r--r-- 1 testuser testuser    9216 Feb 16 22:19 System.Security.dll
-rw-r--r-- 1 testuser testuser    7680 Feb 16 22:19 System.ServiceModel.Web.dll
-rw-r--r-- 1 testuser testuser    6656 Feb 16 22:19 System.ServiceProcess.dll
-rw-r--r-- 1 testuser testuser  855552 Feb 16 22:19 System.Text.Encoding.CodePages.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Text.Encoding.Extensions.dll
-rw-r--r-- 1 testuser testuser    6656 Feb 16 22:19 System.Text.Encoding.dll
-rw-r--r-- 1 testuser testuser   86528 Feb 16 22:19 System.Text.Encodings.Web.dll
-rw-r--r-- 1 testuser testuser  858112 Feb 16 22:19 System.Text.Json.dll
-rw-r--r-- 1 testuser testuser  506880 Feb 16 22:19 System.Text.RegularExpressions.dll
-rw-r--r-- 1 testuser testuser  107520 Feb 16 22:19 System.Threading.Channels.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Threading.Overlapped.dll
-rw-r--r-- 1 testuser testuser  468992 Feb 16 22:19 System.Threading.Tasks.Dataflow.dll
-rw-r--r-- 1 testuser testuser    6656 Feb 16 22:19 System.Threading.Tasks.Extensions.dll
-rw-r--r-- 1 testuser testuser   98816 Feb 16 22:19 System.Threading.Tasks.Parallel.dll
-rw-r--r-- 1 testuser testuser    7680 Feb 16 22:19 System.Threading.Tasks.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Threading.Thread.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Threading.ThreadPool.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Threading.Timer.dll
-rw-r--r-- 1 testuser testuser   69632 Feb 16 22:19 System.Threading.dll
-rw-r--r-- 1 testuser testuser  337920 Feb 16 22:19 System.Transactions.Local.dll
-rw-r--r-- 1 testuser testuser    7680 Feb 16 22:19 System.Transactions.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.ValueTuple.dll
-rw-r--r-- 1 testuser testuser   42496 Feb 16 22:19 System.Web.HttpUtility.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Web.dll
-rw-r--r-- 1 testuser testuser    6656 Feb 16 22:19 System.Windows.dll
-rw-r--r-- 1 testuser testuser    6656 Feb 16 22:19 System.Xml.Linq.dll
-rw-r--r-- 1 testuser testuser   13312 Feb 16 22:19 System.Xml.ReaderWriter.dll
-rw-r--r-- 1 testuser testuser    7168 Feb 16 22:19 System.Xml.Serialization.dll
-rw-r--r-- 1 testuser testuser    6656 Feb 16 22:19 System.Xml.XDocument.dll
-rw-r--r-- 1 testuser testuser    8192 Feb 16 22:19 System.Xml.XPath.XDocument.dll
-rw-r--r-- 1 testuser testuser    6144 Feb 16 22:19 System.Xml.XPath.dll
-rw-r--r-- 1 testuser testuser    6656 Feb 16 22:19 System.Xml.XmlDocument.dll
-rw-r--r-- 1 testuser testuser    8704 Feb 16 22:19 System.Xml.XmlSerializer.dll
-rw-r--r-- 1 testuser testuser   15872 Feb 16 22:19 System.Xml.dll
-rw-r--r-- 1 testuser testuser   46080 Feb 16 22:19 System.dll
-rw-r--r-- 1 testuser testuser    7168 Feb 16 22:19 WindowsBase.dll
-rw-r--r-- 1 testuser testuser    2505 Feb 16 22:15 coreclr_delegates.h
-rwxr-xr-x 1 testuser testuser   92584 Feb 16 21:54 createdump
-rw-r--r-- 1 testuser testuser   10717 Feb 16 22:15 hostfxr.h
-rw-r--r-- 1 testuser testuser 3671090 Feb 16 21:55 libSystem.IO.Compression.Native.a
-rwxr-xr-x 1 testuser testuser  907640 Feb 16 21:55 libSystem.IO.Compression.Native.so
-rw-r--r-- 1 testuser testuser  425486 Feb 16 21:55 libSystem.Native.a
-rwxr-xr-x 1 testuser testuser   75960 Feb 16 21:55 libSystem.Native.so
-rw-r--r-- 1 testuser testuser   36752 Feb 16 21:55 libSystem.Net.Security.Native.a
-rwxr-xr-x 1 testuser testuser   14536 Feb 16 21:55 libSystem.Net.Security.Native.so
-rw-r--r-- 1 testuser testuser  997210 Feb 16 21:55 libSystem.Security.Cryptography.Native.OpenSsl.a
-rwxr-xr-x 1 testuser testuser  129224 Feb 16 21:55 libSystem.Security.Cryptography.Native.OpenSsl.so
-rwxr-xr-x 1 testuser testuser 3175376 Feb 16 21:54 libclrjit.so
-rwxr-xr-x 1 testuser testuser 7109080 Feb 16 21:53 libcoreclr.so
-rwxr-xr-x 1 testuser testuser  628520 Feb 16 21:46 libcoreclrtraceptprovider.so
-rwxr-xr-x 1 testuser testuser  635568 Feb 16 21:51 libdbgshim.so
-rwxr-xr-x 1 testuser testuser  371112 Feb 16 22:19 libhostpolicy.so
-rwxr-xr-x 1 testuser testuser 2576624 Feb 16 21:53 libmscordaccore.so
-rwxr-xr-x 1 testuser testuser 1840448 Feb 16 21:54 libmscordbi.so
-rw-r--r-- 1 testuser testuser 1715646 Feb 16 22:18 libnethost.a
-rw-r--r-- 1 testuser testuser   48128 Feb 16 21:55 mscorlib.dll
-rw-r--r-- 1 testuser testuser    2843 Feb 16 22:15 nethost.h
-rw-r--r-- 1 testuser testuser  105472 Feb 16 22:19 netstandard.dll

Note If I run the container as root (no USER directive), the dotnet-dump command works successfully. (also dont need to setcap as i'm root)

For information, i'm using image mcr.microsoft.com/dotnet/aspnet:5.0 as final build stage image

mikem8361 commented 3 years ago

As far as I know all you should need is the --cap-add=SYS_PTRACE (or --privileged) when starting the docker container. You shouldn't need the setcap or setting /usr/shared/dotnet/shared to your testuser. I'm not an expect on docker and maybe the 5.0 image is different somehow that what we are using for testing in the diagnostics repo's CI builds.

/cc: @shirhatti

DOMZE commented 3 years ago

@mikem8361 you can easily reproduce the problem using the following:

In a command line:

mkdir DotnetDiagToolsBug
dotnet new mvc
touch Dockerfile

Dockerfile:

#See https://aka.ms/containerfastmode to understand how Visual Studio uses this Dockerfile to build your images for faster debugging.

FROM mcr.microsoft.com/dotnet/aspnet:5.0 AS base
WORKDIR /app
EXPOSE 80
EXPOSE 443

FROM mcr.microsoft.com/dotnet/sdk:5.0 AS build
WORKDIR /src
COPY ["DotnetDiagToolsBug.csproj", "."]
RUN dotnet restore "./DotnetDiagToolsBug.csproj"
COPY . .
WORKDIR "/src/."
RUN dotnet build "DotnetDiagToolsBug.csproj" -c Release -o /app/build

# dotnet tools are currently available as part of SDK so we need to create them in an sdk image
# and copy them to our final runtime image
FROM mcr.microsoft.com/dotnet/sdk:5.0 AS tools-install
RUN dotnet tool install --tool-path /dotnetcore-tools dotnet-trace
RUN dotnet tool install --tool-path /dotnetcore-tools dotnet-dump
RUN dotnet tool install --tool-path /dotnetcore-tools dotnet-gcdump
RUN dotnet tool install --tool-path /dotnetcore-tools dotnet-counters

FROM build AS publish
RUN dotnet publish "DotnetDiagToolsBug.csproj" -c Release -o /app/publish

# add the testuser system group
RUN groupadd --system --gid 10101 testuser
# add the testuser system user, without a password and without a login shell with the testuser group created before
RUN adduser --system --disabled-password --shell /sbin/nologin --home /testuser --uid 10101 --ingroup testuser testuser
# set owner for our source and built app to the created user and group.
RUN chown -R testuser:testuser /app/publish

FROM base AS final
ARG DEBIAN_FRONTEND=noninteractive
WORKDIR /app

# update system and install necessary packages
RUN apt-get update && apt-get install -y \
  libcap2-bin \
  && rm -rf /var/lib/apt/lists/*

COPY --from=publish /app/publish .
COPY --from=publish /etc/group /etc/passwd /etc/
COPY --from=tools-install /dotnetcore-tools /opt/dotnetcore-tools

# set the privileges for dotnet to run on privileged ports
# flags: e=activated , p=permitted
RUN dotnetBinary=$(which dotnet) && setcap cap_net_bind_service=+ep $(readlink -f $dotnetBinary)
ENV PATH "$PATH:/opt/dotnetcore-tools"

USER testuser
ENTRYPOINT ["dotnet", "DotnetDiagToolsBug.dll"]

Build and run the image:

docker build -t dotnettoolsbug .
docker run -p 5000:80 --name dotnettoolsbug --cap-add=SYS_PTRACE dotnettoolsbug

in another window:

docker exec -it dotnettoolsbug bash

in the container:

cd /tmp && dotnet-dump collect --process-id 1

Output:

Writing full to /tmp/core_20210323_194228
Writing dump failed (HRESULT: 0x80004005)

In the app logs output:

Gathering state for process 1 dotnet
ptrace(ATTACH, 1) FAILED Operation not permitted
hoyosjs commented 3 years ago

The ptrace attach part fails because there's no ambient capability transfer to the other user. Once you manually set it at the file level it works, but you'll see there's loading issues due to other CAP issues. Also, the setuid calls used by docker seems to mess the state of the dotnet-process. dotnet-dump won't work as a side car as the one that generates the dump is still the target process. I could not find a way to work around these issues

The dotnet-trace bug is a bit suspicious as I don't know how something would be null here: https://github.com/dotnet/diagnostics/blob/e11fe8401a6c8b67dbfe086acef2a75e1b327e04/src/Tools/dotnet-trace/CommandLine/Commands/CollectCommand.cs#L163. Not sure why things are null but I can't attach a debugger.

DOMZE commented 3 years ago

@hoyosjs what is your recommendation then?

hoyosjs commented 3 years ago

Sorry, I this got buried in notifications @DOMZE. Currently I have no good recommendations. I will be experimenting with user namespaces, at which point a lot of the security concerns from using containers will be mitigated. As for getting full de-escalation from things like yum/dnf/apt installations and port concerns, I don't have any clear guidance. Maybe I'll try to reach out to see what causes this behavior.

bluetentacle commented 3 years ago

We too have long been troubled by this issue. dotnet-dump is effectively useless to us as a production troubleshooting tool, because we cannot run services as root in production.