Add more logs for `ValidateServerCertificate`

[RobertIndie]

Is your feature request related to a problem? Please describe

We encountered an exception:

System.Security.Authentication.AuthenticationException: The remote certificate was rejected by the provided RemoteCertificateValidationCallback.    at DotPulsar.Internal.SubConsumer`1.Guard()

It seems that there may be some errors in ValidateServerCertificate that failed the validation. There are some checks for the certificate. But it didn't print any detailed logs explaining why the check failed. We couldn't find more details about the root cause.

Describe the solution you'd like and alternatives you've considered

Adding more detailed logs in ValidateServerCertificate to explain why the validation failed would be helpful.

[blankensteiner]

Hi @RobertIndie That is indeed a reasonable request, so let's talk about our options. I would have hoped the AuthenticationException would give some more information. There is no data and no inner exception? It would have been preferable that Microsoft had at least included the SslPolicyErrors in the exception. DotPulsar doesn't have any log dependencies. All insights are given via exceptions or callbacks. What is your take on how to give the user the needed information? Regarding the error, have you tried not verifying the certificate authority and/or name? And not checking the certificate revocation list?

[RobertIndie]

I think it's fine to provide information to the user through the AuthenticationException.

How about using a lambda for ValidateServerCertificate to pass out some error information? We can then include this information in the AuthenticationException and throw it to the users.

Regarding the error, have you tried not verifying the certificate authority and/or name? And not checking the certificate revocation list?

From the current signs, it seems that the problem is caused by an error when the client accesses the Lets Encrypt revocation list service, leading to a revocation check failure.

[blankensteiner]

Hi @RobertIndie, the issue is that we do not throw the exception and therefore can't pass information that way. When you inspect the exception, there is no data and no inner exception?

[entvex]

@RobertIndie Did you have time to inspect the inner exception?

[RobertIndie]

there is no data and no inner exception?

Haven't reproduced it yet. I don't think there is an inner exception for it. If we return false anywhere in ValidateServerCertificate, an error like the one below will occur, which doesn't contain any inner exception.

DotPulsar.Exceptions.ProducerFaultedException
Producer has faulted
   at DotPulsar.Internal.Producer`1.ChoosePartitions(MessageMetadata metadata, CancellationToken cancellationToken) in /Users/rbt/code/pulsar-dotpulsar/src/DotPulsar/Internal/Producer.cs:line 208
   at DotPulsar.Internal.Producer`1.InternalSend(MessageMetadata metadata, TMessage message, Boolean sendOpCancelable, TaskCompletionSource`1 tcs, Func`2 onMessageSent, Action`1 onFailed, CancellationToken cancellationToken) in /Users/rbt/code/pulsar-dotpulsar/src/DotPulsar/Internal/Producer.cs:line 272
   at DotPulsar.Internal.Producer`1.Send(MessageMetadata metadata, TMessage message, CancellationToken cancellationToken) in /Users/rbt/code/pulsar-dotpulsar/src/DotPulsar/Internal/Producer.cs:line 234
   at DotPulsar.Extensions.SendExtensions.Send[TMessage](ISend`1 sender, TMessage message, CancellationToken cancellationToken) in /Users/rbt/code/pulsar-dotpulsar/src/DotPulsar/Extensions/SendExtensions.cs:line 59
   at DotPulsar.Tests.MessageIdTests.Test() in /Users/rbt/code/pulsar-dotpulsar/tests/DotPulsar.Tests/MessageIdTests.cs:line 37
   at Xunit.Sdk.TestInvoker`1.<>c__DisplayClass46_0.<<InvokeTestMethodAsync>b__1>d.MoveNext() in /_/src/xunit.execution/Sdk/Frameworks/Runners/TestInvoker.cs:line 253
--- End of stack trace from previous location ---
   at Xunit.Sdk.ExecutionTimer.AggregateAsync(Func`1 asyncAction) in /_/src/xunit.execution/Sdk/Frameworks/ExecutionTimer.cs:line 48
   at Xunit.Sdk.ExceptionAggregator.RunAsync(Func`1 code) in /_/src/xunit.core/Sdk/ExceptionAggregator.cs:line 90

System.Security.Authentication.AuthenticationException
The remote certificate was rejected by the provided RemoteCertificateValidationCallback.
   at System.Net.Security.SslStream.SendAuthResetSignal(ReadOnlySpan`1 alert, ExceptionDispatchInfo exception)
   at System.Net.Security.SslStream.CompleteHandshake(SslAuthenticationOptions sslAuthenticationOptions)
   at System.Net.Security.SslStream.ForceAuthenticationAsync[TIOAdapter](Boolean receiveFirst, Byte[] reAuthenticationData, CancellationToken cancellationToken)
   at DotPulsar.Internal.Connector.EncryptStream(Stream stream, String host, CancellationToken cancellationToken) in /Users/rbt/code/pulsar-dotpulsar/src/DotPulsar/Internal/Connector.cs:line 132
   at DotPulsar.Internal.Connector.EncryptStream(Stream stream, String host, CancellationToken cancellationToken) in /Users/rbt/code/pulsar-dotpulsar/src/DotPulsar/Internal/Connector.cs:line 142
   at DotPulsar.Internal.Connector.Connect(Uri serviceUrl, CancellationToken cancellationToken) in /Users/rbt/code/pulsar-dotpulsar/src/DotPulsar/Internal/Connector.cs:line 58
   at DotPulsar.Internal.ConnectionPool.EstablishNewConnection(PulsarUrl url, CancellationToken cancellationToken) in /Users/rbt/code/pulsar-dotpulsar/src/DotPulsar/Internal/ConnectionPool.cs:line 148
   at DotPulsar.Internal.ConnectionPool.GetConnection(PulsarUrl url, CancellationToken cancellationToken) in /Users/rbt/code/pulsar-dotpulsar/src/DotPulsar/Internal/ConnectionPool.cs:line 143
   at DotPulsar.Internal.ConnectionPool.GetConnection(Uri serviceUrl, CancellationToken cancellationToken) in /Users/rbt/code/pulsar-dotpulsar/src/DotPulsar/Internal/ConnectionPool.cs:line 136
   at DotPulsar.Internal.ConnectionPool.FindConnectionForTopic(String topic, CancellationToken cancellationToken) in /Users/rbt/code/pulsar-dotpulsar/src/DotPulsar/Internal/ConnectionPool.cs:line 82
   at DotPulsar.Internal.ConnectionPool.GetNumberOfPartitions(String topic, CancellationToken cancellationToken) in /Users/rbt/code/pulsar-dotpulsar/src/DotPulsar/Internal/ConnectionPool.cs:line 224
   at DotPulsar.Internal.Producer`1.Monitor() in /Users/rbt/code/pulsar-dotpulsar/src/DotPulsar/Internal/Producer.cs:line 108
   at DotPulsar.Internal.Executor.TryExecuteOnce(Func`1 func, CancellationToken cancellationToken) in /Users/rbt/code/pulsar-dotpulsar/src/DotPulsar/Internal/Executor.cs:line 97
   at DotPulsar.Internal.Executor.TryExecuteOnce(Func`1 func, CancellationToken cancellationToken) in /Users/rbt/code/pulsar-dotpulsar/src/DotPulsar/Internal/Executor.cs:line 103
   at DotPulsar.Internal.Executor.Execute(Func`1 func, CancellationToken cancellationToken) in /Users/rbt/code/pulsar-dotpulsar/src/DotPulsar/Internal/Executor.cs:line 41
   at DotPulsar.Internal.Producer`1.Setup() in /Users/rbt/code/pulsar-dotpulsar/src/DotPulsar/Internal/Producer.cs:line 94

I think we need to provide more information about the specific cause of the error when we return false in ValidateServerCertificate. That's why I suggest using a lambda to pass this information. Then, we can catch the exception here and add more details to the AuthenticationException.

[entvex]

@RobertIndie We have talked about it internally and we may have a solution. Please try https://www.nuget.org/packages/DotPulsar/3.4.0-rc.1 https://github.com/apache/pulsar-dotpulsar/releases/tag/3.4.0-rc.1

[RobertIndie]

We have talked about it internally and we may have a solution. Please try https://www.nuget.org/packages/DotPulsar/3.4.0-rc.1 https://github.com/apache/pulsar-dotpulsar/releases/tag/3.4.0-rc.1

Thanks. That looks good to me.

[entvex]

Closing. The new 3.4.0 with the fix for this will release soon.