search

dotnet / sdk-container-builds

Libraries and build tooling to create container images from .NET projects using MSBuild
https://learn.microsoft.com/en-us/dotnet/core/docker/publish-as-container
MIT License
176 stars 30 forks source link

Native container support does not respect system proxy settings #467

Open deeprobin opened 1 year ago

deeprobin commented 1 year ago

Describe the bug

I want to publish a Docker image behind a corporate proxy using dotnet publish.

I had to find out, however, that the proxy settings (via for example) Windows are not taken into account.

Since many larger companies use a proxy to regulate the traffic, I will not be the only one who has this problem.

I am aware that you can set HTTP_PROXY and HTTPS_PROXY with the proxy address as the default environment variable in the docker configuration, but in my opinion this is also not the best choice for security reasons, since the credentials have to be passed as plain text and are not stored in the Windows Credential Manager.

I would like the default system proxy to be considered for the build of the Docker image.

To Reproduce

  1. Create a project with native container support

    <PropertyGroup>
<EnableSdkContainerSupport>true</EnableSdkContainerSupport>
</PropertyGroup>

  2. Setup a proxy / use a corporate proxy

  3. Work behind a firewall, which blocks any traffic that does not go through the proxy

  4. dotnet publish --os linux --arch x64 /t:PublishContainer -c Release

Exceptions (if any)

Error MSB4081: The "CreateNewImage" task failed unexpectedly. Timeout to mcr.microsoft.com:443

C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018: The "CreateNewImage" task failed unexpectedly. [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018: System.AggregateException: One or more errors occurred. (A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. (mcr.microsoft.com:443)) [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:  ---> System.Net.Http.HttpRequestException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. (mcr.microsoft.com:443) [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:  ---> System.Net.Sockets.SocketException (10060): A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken) [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token) [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|281_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken) [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    --- End of inner exception stack trace --- [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken) [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(QueueItem queueItem) [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken) [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at System.Net.Http.HttpConnectionPool.HttpConnectionWaiter`1.WaitForConnectionAsync(Boolean async, CancellationToken requestCancellationToken) [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken) [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken) [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at Microsoft.NET.Build.Containers.AuthHandshakeMessageHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) in D:\a\_work\1\s\Microsoft.NET.Build.Containers\AuthHandshakeMessageHandler.cs:line 173 [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken) [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at Microsoft.NET.Build.Containers.Registry.GetManifest(String repositoryName, String reference) in D:\a\_work\1\s\Microsoft.NET.Build.Containers\Registry.cs:line 136 [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at Microsoft.NET.Build.Containers.Registry.GetImageManifest(String repositoryName, String reference, String runtimeIdentifier, String runtimeIdentifierGraphPath) in D:\a\_work\1\s\Microsoft.NET.Build.Containers\Registry.cs:line 100 [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    --- End of inner exception stack trace --- [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification) [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at Microsoft.NET.Build.Containers.Tasks.CreateNewImage.GetBaseImage() in D:\a\_work\1\s\Microsoft.NET.Build.Containers\CreateNewImage.cs:line 81 [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at Microsoft.NET.Build.Containers.Tasks.CreateNewImage.Execute() in D:\a\_work\1\s\Microsoft.NET.Build.Containers\CreateNewImage.cs:line 97 [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]
C:\...\build\Microsoft.NET.Build.Containers.targets(114,9): error MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask) [F:\...\AgentDirectory\_work\20\s\PrintAPI\src\PrintAPI.Host\PrintAPI.Host.csproj]

It should be possible to reproduce this bug. If necessary, I can send a binary log.

Further technical details

.NET SDKs installed:
  7.0.202 [C:\Program Files\dotnet\sdk]
  7.0.203 [C:\Program Files\dotnet\sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.AspNetCore.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 5.0.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.NETCore.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
  Microsoft.WindowsDesktop.App 5.0.10 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 6.0.16 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.0 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.4 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]
  Microsoft.WindowsDesktop.App 7.0.5 [C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App]

Not relevant, but VS Professional 2022 / Latest version

baronfel commented 1 year ago

This is an interesting report. the SocketsHttpHandler that we use by default has UseProxy set to true, which then uses the default IWebProxy to actually proxy requests. From the docs:

The default instance returned by this property will initialize following a different set of rules depending on your platform:

For Windows: Reads proxy configuration from environment variables or, if those are not defined, from the user's proxy settings.

For macOS: Reads proxy configuration from environment variables or, if those are not defined, from the system's proxy settings.

For Linux: Reads proxy configuration from environment variables or, in case those are not defined, this property initializes a non-configured instance that bypasses all addresses.

This sounds like exactly the behavior I'd expect.

deeprobin commented 1 year ago

In what way can I provide more information so we can find the root cause?

baronfel commented 10 months ago

I'm not very well-versed with proxy configuration on HttpClient myself, but the API docs for this are here - you may be able to write a small utility to investigate and compare. My belief based on docs is that the .NET runtime should be using the user-level proxy settings configuration already, but I don't have one set up and so cannot easily verify this.