truecharts / public

Community Helm Chart Repository
https://truecharts.org
GNU Affero General Public License v3.0
1.15k stars 618 forks source link

Duplicati with Backblaze B2 - certificate error from mono #2317

Closed redakula closed 2 years ago

redakula commented 2 years ago

App Name

Duplicati

SCALE Version

22.02.0

App Version

5.0.6

Application Events

The error is within duplicati

Application Logs

System.Net.WebException: Error: TrustFailure (Authentication failed, see inner exception.) ---> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception. ---> Mono.Btls.MonoBtlsException: Ssl error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED
  at /build/mono-6.12.0.107/external/boringssl/ssl/handshake_client.c:1132
  at Mono.Btls.MonoBtlsContext.ProcessHandshake () [0x00048] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake (Mono.Net.Security.AsyncOperationStatus status, System.Boolean renegotiate) [0x000da] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at (wrapper remoting-invoke-with-check) Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake(Mono.Net.Security.AsyncOperationStatus,bool)
  at Mono.Net.Security.AsyncHandshakeRequest.Run (Mono.Net.Security.AsyncOperationStatus status) [0x00006] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at Mono.Net.Security.AsyncProtocolRequest.ProcessOperation (System.Threading.CancellationToken cancellationToken) [0x000fc] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
   --- End of inner exception stack trace ---
  at Mono.Net.Security.MobileAuthenticatedStream.ProcessAuthentication (System.Boolean runSynchronously, Mono.Net.Security.MonoSslAuthenticationOptions options, System.Threading.CancellationToken cancellationToken) [0x00262] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at Mono.Net.Security.MonoTlsStream.CreateStream (System.Net.WebConnectionTunnel tunnel, System.Threading.CancellationToken cancellationToken) [0x0016a] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at System.Net.WebConnection.CreateStream (System.Net.WebOperation operation, System.Boolean reused, System.Threading.CancellationToken cancellationToken) [0x001ba] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
   --- End of inner exception stack trace ---
  at System.Net.WebConnection.CreateStream (System.Net.WebOperation operation, System.Boolean reused, System.Threading.CancellationToken cancellationToken) [0x0021a] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at System.Net.WebConnection.InitConnection (System.Net.WebOperation operation, System.Threading.CancellationToken cancellationToken) [0x00141] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at System.Net.WebOperation.Run () [0x0009a] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at System.Net.WebCompletionSource`1[T].WaitForCompletion () [0x00094] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at System.Net.HttpWebRequest.RunWithTimeoutWorker[T] (System.Threading.Tasks.Task`1[TResult] workerTask, System.Int32 timeout, System.Action abort, System.Func`1[TResult] aborted, System.Threading.CancellationTokenSource cts) [0x000f8] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at Duplicati.Library.Utility.AsyncHttpRequest+AsyncWrapper.GetResponseOrStream () [0x0004d] in <2a3ee711c7c04f6c957360f2cf183a7f>:0 
  at Duplicati.Library.Utility.AsyncHttpRequest.GetResponse () [0x00044] in <2a3ee711c7c04f6c957360f2cf183a7f>:0 
  at Duplicati.Library.Backend.Backblaze.B2AuthHelper.get_Config () [0x0013d] in <f30a9ba7585445e094ae4320fb244dfc>:0 
  at Duplicati.Library.Backend.Backblaze.B2AuthHelper.get_APIUrl () [0x00000] in <f30a9ba7585445e094ae4320fb244dfc>:0 
  at Duplicati.Library.Backend.Backblaze.B2.List () [0x00011] in <f30a9ba7585445e094ae4320fb244dfc>:0 
  at Duplicati.Library.Interface.BackendExtensions.TestList (Duplicati.Library.Interface.IBackend backend) [0x00000] in <fd3642a459884bd9a2412b4eda050109>:0 
  at Duplicati.Library.Backend.Backblaze.B2.Test () [0x00000] in <f30a9ba7585445e094ae4320fb244dfc>:0 
  at Duplicati.Server.WebServer.RESTMethods.RemoteOperation.TestConnection (System.String url, Duplicati.Server.WebServer.RESTMethods.RequestInfo info) [0x000b7] in <156011ea63b34859b4073abdbf0b1573>:0 
  at Duplicati.Server.WebServer.RESTMethods.RemoteOperation.POST (System.String key, Duplicati.Server.WebServer.RESTMethods.RequestInfo info) [0x00094] in <156011ea63b34859b4073abdbf0b1573>:0 
  at Duplicati.Server.WebServer.RESTHandler.DoProcess (Duplicati.Server.WebServer.RESTMethods.RequestInfo info, System.String method, System.String module, System.String key) [0x00289] in <156011ea63b34859b4073abdbf0b1573>:0

Application Configuration

All settings are at their defaults

Describe the bug

During configuration of the backups to backblaze B2 when you test the connection the test failes with the above error.

To Reproduce

Try to configure a backup destination on backblaze B2

Expected Behavior

Expect the test and backblaze to work

Screenshots

N/A

Additional Context

The issue appears to be the one referenced here: https://forum.duplicati.com/t/http-send-report-errors-duplicati-monitoring/13157/38

Basically it appears a certificate in the chain (DST Root CA X3) is expired so either mono or the certificates need to be updated.

I've read and agree with the following

stavros-k commented 2 years ago

You answered your own problem there.. The problem is within duplicati.

We don't build the container. We just wrap it in a helm chart.

What I can do, is update the digest pin of the image we use. If they have included a fix in there, you are lucky. Otherwise you have to ask them to fix it.

Expect the app update in couple of hours. version 5.0.7.

Closing this as we can't do anything else here.

stavros-k commented 2 years ago

Also next time, actually provide ALL configurations / Application Events and Logs in FULL. Even if you don't see errors or if everything is default.

redakula commented 2 years ago

Hi Nice with quick updates :) Deleted the app and installed the new version. No difference in the result - the certificate error above still occurs... As i read the referenced thread the issue is in the mono version in the pod having an expired certificate in the chain.

A fix for debian is provided here but i have never used pods before so i am unsure of how to apply it to this case. https://forum.duplicati.com/t/http-send-report-errors-duplicati-monitoring/13157/16

There is actually additional errors about certificates for an update process as well:

28 Mar 2022 16:39: Reporting error gave error
System.ObjectDisposedException: Can not write to a closed TextWriter.
  at System.IO.StreamWriter.Flush (System.Boolean flushStream, System.Boolean flushEncoder) [0x00008] in <d13c8b563008422a8c5aaec0a74089cc>:0 
  at System.IO.StreamWriter.Flush () [0x00006] in <d13c8b563008422a8c5aaec0a74089cc>:0 
  at Duplicati.Server.WebServer.RESTHandler.DoProcess (Duplicati.Server.WebServer.RESTMethods.RequestInfo info, System.String method, System.String module, System.String key) [0x003bc] in <156011ea63b34859b4073abdbf0b1573>:0 
28 Mar 2022 16:37: Error in updater
System.Net.WebException: Error: TrustFailure (Authentication failed, see inner exception.) ---> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception. ---> Mono.Btls.MonoBtlsException: Ssl error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED
  at /build/mono-6.12.0.107/external/boringssl/ssl/handshake_client.c:1132
  at Mono.Btls.MonoBtlsContext.ProcessHandshake () [0x00048] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake (Mono.Net.Security.AsyncOperationStatus status, System.Boolean renegotiate) [0x000da] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at (wrapper remoting-invoke-with-check) Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake(Mono.Net.Security.AsyncOperationStatus,bool)
  at Mono.Net.Security.AsyncHandshakeRequest.Run (Mono.Net.Security.AsyncOperationStatus status) [0x00006] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at Mono.Net.Security.AsyncProtocolRequest.ProcessOperation (System.Threading.CancellationToken cancellationToken) [0x000fc] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
   --- End of inner exception stack trace ---
  at Mono.Net.Security.MobileAuthenticatedStream.ProcessAuthentication (System.Boolean runSynchronously, Mono.Net.Security.MonoSslAuthenticationOptions options, System.Threading.CancellationToken cancellationToken) [0x00262] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at Mono.Net.Security.MonoTlsStream.CreateStream (System.Net.WebConnectionTunnel tunnel, System.Threading.CancellationToken cancellationToken) [0x0016a] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at System.Net.WebConnection.CreateStream (System.Net.WebOperation operation, System.Boolean reused, System.Threading.CancellationToken cancellationToken) [0x001ba] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
   --- End of inner exception stack trace ---
  at System.Net.WebConnection.CreateStream (System.Net.WebOperation operation, System.Boolean reused, System.Threading.CancellationToken cancellationToken) [0x0021a] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at System.Net.WebConnection.InitConnection (System.Net.WebOperation operation, System.Threading.CancellationToken cancellationToken) [0x00141] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at System.Net.WebOperation.Run () [0x0009a] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at System.Net.WebCompletionSource`1[T].WaitForCompletion () [0x00094] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at System.Net.HttpWebRequest.RunWithTimeoutWorker[T] (System.Threading.Tasks.Task`1[TResult] workerTask, System.Int32 timeout, System.Action abort, System.Func`1[TResult] aborted, System.Threading.CancellationTokenSource cts) [0x000f8] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at System.Net.HttpWebRequest.GetResponse () [0x00016] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at System.Net.WebClient.GetWebResponse (System.Net.WebRequest request) [0x00000] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at System.Net.WebClient.DownloadBits (System.Net.WebRequest request, System.IO.Stream writeStream) [0x000e6] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at System.Net.WebClient.DownloadFile (System.Uri address, System.String fileName) [0x00088] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at System.Net.WebClient.DownloadFile (System.String address, System.String fileName) [0x00008] in <6bc04dcac0a443ee834a449c98b8ed9d>:0 
  at (wrapper remoting-invoke-with-check) System.Net.WebClient.DownloadFile(string,string)
  at Duplicati.Library.AutoUpdater.UpdaterManager.CheckForUpdate (Duplicati.Library.AutoUpdater.ReleaseType channel) [0x000ee] in <8d4cb1693e00483189d3952c3f0ed20f>:0 

Sorry about the missing config - here is the configuration:

Application Name: duplicati
    Version: 5.0.9
    :
        Show Advanced Controller Settings: false
        Show Expert Configuration Options: false
    Timezone: 'Europe/Copenhagen' timezone
    Show Expert Config: false
    Configure Service(s):
        Main Service:
            Service Type: Simple
            Service's Port(s) Configuration:
                Main Service Port Configuration:
                    Port: 8200
                    Show Advanced settings: false
    Show Expert Config: false
    Integrated Persistent Storage:
        App Config Storage:
            Type of Storage: PVC (simple)
            readOnly: false
            Show Advanced Options: false
    Additional app storage: 1
    :
        Main Ingress:
            Enable Ingress: false
            Show Expert Configuration Options: false
    Container Security Settings:
        Change PUID / UMASK values: false
    Show Advanced Security Settings: false
    Pod Security Context:
        runAsUser: 568
        runAsGroup: 568
        fsGroup: 568
        When should we take ownership?: OnRootMismatch
    Set Custom Resource Limits/Requests (Advanced): false
    :
        VPN:
            Type: disabled
        Codeserver:
            enabled: false
        Promtail:
            enabled: false
        Netshoot:
            enabled: false
    (Advanced) Horizontal Pod Autoscaler:
        enabled: false
    (Advanced) Network Policy:
        enabled: false
stavros-k commented 2 years ago

As I already said, the mono version or whatever the problem is, is contained in the image THEY provide. It's not something we can fix.

TopicsLP commented 2 years ago

Sorry to reply on a closed issue, but i have a similar problem with Let's Encrypt certificates, and i think i got some helpful information. In my TrueNAS Scale the docker images shows a "CREATED" time of "12 months ago"

root@TrueNasScale[~]# docker images tccr.io/truecharts/duplicati -a
REPOSITORY                     TAG       IMAGE ID       CREATED         SIZE
tccr.io/truecharts/duplicati   <none>    e4ab3b762518   12 months ago   709MB

Version Information from TrueNAS>Apps

duplicati / latest_6.0.5
tccr.io/truecharts/duplicati:latest@sha256:9435ca54cf320b8f6b285e4bb6b304e285e828a2b97f29f3037ac604924d99a0Up to date

So i tested with a normal Docker Container from the Linuxserver.io Team. I did encounter the same issue in version: lscr.io/linuxserver/duplicati:v2.0.6.3-2.0.6.3_beta_2021-06-17-ls102 (created 2021-06-17)

I did then test a newer random version, and the issue did not occur: lscr.io/linuxserver/duplicati:v2.0.6.3-2.0.6.3_beta_2021-06-17-ls131 (created 2022-04-22)

I assume even if the TrueCharts repo got updated 3 months ago (what i found) somewhere inside of TrueCharts there is somewhere an old image for the docker container.

Error Messages from the Updater inside of Duplicati (just for testing):

``` System.Net.WebException: Error: TrustFailure (Authentication failed, see inner exception.) ---> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception. ---> Mono.Btls.MonoBtlsException: Ssl error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED at /build/mono-5.20.1.34/external/boringssl/ssl/handshake_client.c:1132 at Mono.Btls.MonoBtlsContext.ProcessHandshake () [0x00038] in <91935ad653254a93b9d73a9f8f2f7a2d>:0 at Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake (Mono.Net.Security.AsyncOperationStatus status, System.Boolean renegotiate) [0x000a1] in <91935ad653254a93b9d73a9f8f2f7a2d>:0 at (wrapper remoting-invoke-with-check) Mono.Net.Security.MobileAuthenticatedStream.ProcessHandshake(Mono.Net.Security.AsyncOperationStatus,bool) at Mono.Net.Security.AsyncHandshakeRequest.Run (Mono.Net.Security.AsyncOperationStatus status) [0x00006] in <91935ad653254a93b9d73a9f8f2f7a2d>:0 at Mono.Net.Security.AsyncProtocolRequest.ProcessOperation (System.Threading.CancellationToken cancellationToken) [0x000ff] in <91935ad653254a93b9d73a9f8f2f7a2d>:0 at Mono.Net.Security.AsyncProtocolRequest.StartOperation (System.Threading.CancellationToken cancellationToken) [0x0008b] in <91935ad653254a93b9d73a9f8f2f7a2d>:0 --- End of inner exception stack trace --- at Mono.Net.Security.MobileAuthenticatedStream.ProcessAuthentication (System.Boolean runSynchronously, Mono.Net.Security.MonoSslAuthenticationOptions options, System.Threading.CancellationToken cancellationToken) [0x00252] in <91935ad653254a93b9d73a9f8f2f7a2d>:0 at Mono.Net.Security.MonoTlsStream.CreateStream (System.Net.WebConnectionTunnel tunnel, System.Threading.CancellationToken cancellationToken) [0x00126] in <91935ad653254a93b9d73a9f8f2f7a2d>:0 at System.Net.WebConnection.CreateStream (System.Net.WebOperation operation, System.Boolean reused, System.Threading.CancellationToken cancellationToken) [0x001ba] in <91935ad653254a93b9d73a9f8f2f7a2d>:0 --- End of inner exception stack trace --- at System.Net.WebConnection.CreateStream (System.Net.WebOperation operation, System.Boolean reused, System.Threading.CancellationToken cancellationToken) [0x0021a] in <91935ad653254a93b9d73a9f8f2f7a2d>:0 at System.Net.WebConnection.InitConnection (System.Net.WebOperation operation, System.Threading.CancellationToken cancellationToken) [0x00141] in <91935ad653254a93b9d73a9f8f2f7a2d>:0 at System.Net.WebOperation.Run () [0x0009a] in <91935ad653254a93b9d73a9f8f2f7a2d>:0 at System.Net.WebCompletionSource`1[T].WaitForCompletion () [0x00094] in <91935ad653254a93b9d73a9f8f2f7a2d>:0 at System.Net.HttpWebRequest.RunWithTimeoutWorker[T] (System.Threading.Tasks.Task`1[TResult] workerTask, System.Int32 timeout, System.Action abort, System.Func`1[TResult] aborted, System.Threading.CancellationTokenSource cts) [0x000f8] in <91935ad653254a93b9d73a9f8f2f7a2d>:0 at System.Net.HttpWebRequest.GetResponse () [0x00016] in <91935ad653254a93b9d73a9f8f2f7a2d>:0 at System.Net.WebClient.GetWebResponse (System.Net.WebRequest request) [0x00000] in <91935ad653254a93b9d73a9f8f2f7a2d>:0 at System.Net.WebClient.DownloadBits (System.Net.WebRequest request, System.IO.Stream writeStream) [0x000e6] in <91935ad653254a93b9d73a9f8f2f7a2d>:0 at System.Net.WebClient.DownloadFile (System.Uri address, System.String fileName) [0x00088] in <91935ad653254a93b9d73a9f8f2f7a2d>:0 at System.Net.WebClient.DownloadFile (System.String address, System.String fileName) [0x00008] in <91935ad653254a93b9d73a9f8f2f7a2d>:0 at (wrapper remoting-invoke-with-check) System.Net.WebClient.DownloadFile(string,string) at Duplicati.Library.AutoUpdater.UpdaterManager.CheckForUpdate (Duplicati.Library.AutoUpdater.ReleaseType channel) [0x000ee] in <8d4cb1693e00483189d3952c3f0ed20f>:0 ```
stavros-k commented 2 years ago

We are not using the LSIO image, we use the official duplicati image. And looks like they didn't release any image with tag latest (or any not canary tags) within the last year . canary had some releases, which is the dev/test channel and I'm not going to use that. Even their "beta/latest" is unstable. You have to raise your issue to duplicati devs, to release a newer release.

PrivatePuffin commented 2 years ago

Also on this: Don't necro issues when we already concluded it was not something we handle, including a clear reference why that's the case.

You're basically wasting everyones time doing so.