search

aws / aws-sdk-net

The official AWS SDK for .NET. For more information on the AWS SDK for .NET, see our web site:
http://aws.amazon.com/sdkfornet/
Apache License 2.0
2.07k stars 862 forks source link

Updating from .NET Core 2.1 to .NET Core 3.1: AmazonIoTDataClient Publish no longer works #1982

Closed JustineAlires closed 11 months ago

JustineAlires commented 2 years ago

Description

After an upgrade from .NET Core 2.1 to .NET Core 3.1, my C# Lambda for publishing mqtt messages no longer works and fails on the Publish Request. Here is the code

private IAmazonIotData CreateAmazonIotDataClient()
{
            var mqttBrokerEndpoint = Environment.GetEnvironmentVariable("MQTTBroker");
            var clientConfig = new AmazonIotDataConfig { ServiceURL = mqttBrokerEndpoint };
            return new AmazonIotDataClient(clientConfig);
}

using (var iotDataClient = CreateAmazonIotDataClient())
 {
          var mqttBrokerEndpoint = Environment.GetEnvironmentVariable("MQTTBroker");
          context.Logger.LogLine($"Created an AmazonIotDataClient: MQTT Env Variable {mqttBrokerEndpoint}");
          var serviceURL = iotDataClient.Config.ServiceURL;
          context.Logger.LogLine($"AmazonIotDataClient: ServiceURL {serviceURL}");
          context.Logger.LogLine($"iotDataClient Config: ProxyHost {iotDataClient.Config.ProxyHost}");
          context.Logger.LogLine($"iotDataClient Config: ProxyPort {iotDataClient.Config.ProxyPort}");
          context.Logger.LogLine($"iotDataClient Config: AllowAutoRedirect {iotDataClient.Config.AllowAutoRedirect}");
          context.Logger.LogLine($"iotDataClient Config: AuthenticationServiceName {iotDataClient.Config.AuthenticationServiceName}");
          context.Logger.LogLine($"iotDataClient Config: Url {iotDataClient.Config.DetermineServiceURL()}");

          // Tried validating the configuration
           iotDataClient.Config.Validate();
           try
           {
                  await iotDataClient.PublishAsync(new PublishRequest
                   {
                                Topic = $"{latestMessageFromMTT.ClientID}/JobDiscussion",
                                Qos = 1,
                                Payload = new MemoryStream(
                                    Encoding.UTF8.GetBytes(JsonSerializer.Serialize(jobDiscussion)))
                            }, cancellationTokenSource.Token);
                   }
                   return new APIGatewayHttpApiV2ProxyResponse
                    {
                        StatusCode = 201,
                        Headers = CorsHeaders
                    };
               }
              catch(Exception e)
               {
                            context.Logger.LogLine($"ERROR: PublishAsync throws Exception: Type: {e.GetType()} Message: {e.ToString()}");
                            return new APIGatewayHttpApiV2ProxyResponse { StatusCode = 500, Headers = CorsHeaders };
                }

Certificates and AWS environment checked

Once I experienced this error I thoroughly checked

To Prove that this error was not my certs or AWS environment, I rewrote my PostJobDiscussion lambda in nodejs and it works perfectly in the same environment.

The goal is to see the MQTT messages come thru. With the .NET 3.1 version I do not get the messages come thru but with the nodejs lambda I see the MQTT messages come thru.

Reproduction Steps

Logs

The Publish fails with the following error ERROR: PublishAsync throws Exception: Type: System.Net.Http.HttpRequestException Message: System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception. ---> System.Security.Authentication.AuthenticationException: The remote certificate is invalid according to the validation procedure. at System.Net.Security.SslStream.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception) at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.StartReceiveBlob(Byte[] buffer, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest) at System.Net.Security.SslStream.PartialFrameCallback(AsyncProtocolRequest asyncRequest) --- End of stack trace from previous location where exception was thrown --- at System.Net.Security.SslStream.ThrowIfExceptional() at System.Net.Security.SslStream.InternalEndProcessAuthentication(LazyAsyncResult lazyResult) at System.Net.Security.SslStream.EndProcessAuthentication(IAsyncResult result) at System.Net.Security.SslStream.EndAuthenticateAsClient(IAsyncResult asyncResult) at System.Net.Security.SslStream.<>c.b__65_1(IAsyncResult iar) at System.Threading.Tasks.TaskFactory1.FromAsyncCoreLogic(IAsyncResult iar, Func2 endFunction, Action1 endAction, Task1 promise, Boolean requiresSynchronization) --- End of stack trace from previous location where exception was thrown --- at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken) --- End of inner exception stack trace --- at System.Net.Http.ConnectHelper.EstablishSslConnectionAsyncCore(Stream stream, SslClientAuthenticationOptions sslOptions, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean allowHttp2, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.GetHttpConnectionAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpConnectionPool.SendWithRetryAsync(HttpRequestMessage request, Boolean doRequestAuth, CancellationToken cancellationToken) at System.Net.Http.RedirectHandler.SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) at System.Net.Http.HttpClient.FinishSendAsyncUnbuffered(Task1 sendTask, HttpRequestMessage request, CancellationTokenSource cts, Boolean disposeCts) at Amazon.Runtime.HttpWebRequestMessage.GetResponseAsync(CancellationToken cancellationToken) at Amazon.Runtime.Internal.HttpHandler1.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.Unmarshaller.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.ErrorHandler.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.EndpointDiscoveryHandler.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeAsync[T](IExecutionContext executionContext) at Amazon.Runtime.Internal.MetricsHandler.InvokeAsync[T](IExecutionContext executionContext) at Jobs.Function.PostJobDiscussion(APIGatewayProxyRequest apiGatewayProxyRequest, ILambdaContext context) in /codebuild/output/src879162451/src/api/Jobs/src/Jobs/PostJobDiscussion.cs:line 125

Environment

Resolution

I have gone thru the .NET Core 2.1 to 3.1 Upgrade guides. I would like to continue to use .NET Core 3.1 for my server side lambdas but until I resolve this issue I will have to rewrite them in nodejs.

I have not been able to find anything on stackoverflow and aws forums concerning this issue. There are several issues I troubleshooted but the certificates I am relying on are valid, active and have not expired.

I did see the open issue from Nov 2020 https://github.com/aws/aws-sdk-net/issues/1739 but I can't imagine that no one else is using .NET Core 3.1 with AWSSDK.IoTData. In Visual Studio I clearly see the Publish method of the dataclient.

This is a :bug: bug-report

ashishdhingra commented 2 years ago

Hi @JustineAlires,

Good morning.

Thanks for opening the issue. Upon investigating at high level, looks like the code for AmazonIotDataClient is auto-generated, refer IotData. The logs shared by you clearly mentions that the remote certificate is invalid. I'm unsure if the below articles/references might help:

Thanks, Ashish

JustineAlires commented 2 years ago

Hello Ashish,
Thank you for your reply. The SSL error is a catch all and you will see as you google this error that there are many different reasons for this error. What I did was verify my Cloudfront distribution, my route 53 records for validation, and my IoT Certificates were valid and working as expected. I wrote the nodejs lambda that published the mqtt message and I was successful at getting the messages across to my device. Same Certs and same environment. This error only happened after the .NET upgrade to 3.1. AWS no longer supports .NET Core 2.1 in their images so to revert would take modifying the docker container. But .NET Core 2.1 is no longer supported so reverting to .NET Core 2.1 is not an option.

I can work with you to reproduce but this is a total blocker for me and I am currently rewriting these lambdas in nodejs, which was not preferable because all of my lambdas are written in C# .NET Core 3.1. I went thru the upgrade guide but there was no mention of IoT. Did you look at this open issue? https://github.com/aws/aws-sdk-net/issues/1739

Thank you, Justine

ashishdhingra commented 12 months ago

@JustineAlires Good afternoon. Just wanted to check if this is still an issue at your end.

Thanks, Ashish

github-actions[bot] commented 11 months ago

This issue has not received a response in 5 days. If you want to keep this issue open, please just leave a comment below and auto-close will be canceled.