Certifcate error when using proxy (Android and Debian)

[totonymous35]

Hi, i tried to use Keybase with a proxy. I tried both on Android and on Debian.

I get the following error in the log :

...x509: certificate signed by unknown authority

On Android the proxy CA certificate is trusted by the system. Same in Debian. I even authorize TLS MITM (in the config file, i see, disabling-certificate-pinning: true).

So what am i doing wrong ?

[totonymous35]

OK, so now it works for connexions to Keybase API server. But, the problem remains for connexions to other keybase servers (bserver, gregor/chat, mdserver).

[maxtaco]

If you changed the config file, make sure you fully restart the app

[totonymous35]

Yes i fully restart the app or the phone and still get the issue with the other servers so i can't use chat and kbfs behind the proxy. I can log in/out, edit profile, etc. 'cause can request keybase.io.

[maxtaco]

hmmm. not sure why that would be, maybe @ddworken has an idea. It should work but maybe it's an issue with your particular proxy. If you send feedback from the app we can look at the logs.

[totonymous35]

Here an extract of logs:

2019-10-15T10:10:01.457890Z ▶ [DEBU kbfs(BSR) connection.go:786] da74d (CONN BlockServerRemoteGet e897dfa8) RetryNotify attempt 2019-10-15T10:10:01.458172Z ▶ [DEBU kbfs(BSR) connection.go:585] da74e (CONN BlockServerRemoteGet e897dfa8) Connection: dialing transport 2019-10-15T10:10:01.458680Z ▶ [DEBU kbfs(BSR) connection.go:246] da74f (CONNTSPT e7520685) Dialing bserver-0.kbfs.keybase.io:443 2019-10-15T10:10:01.475527Z ▶ [DEBU kbfs(BSR) connection.go:274] da750 (CONNTSPT e7520685) baseConn: 192.168.232.2:58529; Calling Handshake 2019-10-15T10:10:01.485172Z ▶ [WARN kbfs(BSR) connection.go:591] da751 (CONN BlockServerRemoteGet e897dfa8) Connection: error dialing transport: x509: certificate signed by unknown authority 2019-10-15T10:10:01.485400Z ▶ [DEBU kbfs(BSR) connection.go:788] da752 (CONN BlockServerRemoteGet e897dfa8) RetryNotify operation result: x509: certificate signed by unknown authority 2019-10-15T10:10:01.485613Z ▶ [WARN kbfs(BSR) bserver_remote.go:232] da753 BlockServerRemoteGet: connection error: x509: certificate signed by unknown authority; retrying in 2s 2019-10-15T10:10:01.660445Z ▶ [DEBU kbfs(BSR) connection.go:786] da754 (CONN BlockServerRemotePut 8bb1c24b) RetryNotify attempt 2019-10-15T10:10:01.660751Z ▶ [DEBU kbfs(BSR) connection.go:585] da755 (CONN BlockServerRemotePut 8bb1c24b) Connection: dialing transport 2019-10-15T10:10:01.661435Z ▶ [DEBU kbfs(BSR) connection.go:246] da756 (CONNTSPT 97c72001) Dialing bserver-1.kbfs.keybase.io:443 2019-10-15T10:10:01.684547Z ▶ [DEBU kbfs(BSR) connection.go:274] da757 (CONNTSPT 97c72001) baseConn: 192.168.232.2:58530; Calling Handshake 2019-10-15T10:10:01.692740Z ▶ [WARN kbfs(BSR) connection.go:591] da758 (CONN BlockServerRemotePut 8bb1c24b) Connection: error dialing transport: x509: certificate signed by unknown authority 2019-10-15T10:10:01.693032Z ▶ [DEBU kbfs(BSR) connection.go:788] da759 (CONN BlockServerRemotePut 8bb1c24b) RetryNotify operation result: x509: certificate signed by unknown authority 2019-10-15T10:10:01.693296Z ▶ [WARN kbfs(BSR) bserver_remote.go:232] da75a BlockServerRemotePut: connection error: x509: certificate signed by unknown authority; retrying in 2s 2019-10-15T10:10:01.715872Z ▶ [DEBU keybase convloader.go:355] da75b ++Chat: | BackgroundConvLoader: waitForResume: resuming loop 2019-10-15T10:10:01.716610Z ▶ [DEBU keybase convloader.go:366] da75c ++Chat: | BackgroundConvLoader: loop: waiting for job 2019-10-15T10:10:02.093826Z ▶ [DEBU keybase sender.go:1688] da75d ++Chat: | Deliverer: deliverLoop: flushing 1 items from the outbox: uid: xxxxx 2019-10-15T10:10:02.094223Z ▶ [DEBU keybase sender.go:1730] da75e ++Chat: | Deliverer: deliverLoop: failed to send msg: uid: XXXXXXXXXXXXXXX convID: XXXXXXXXX obid: XXXXXXXXXXX err: disconnected from chat server attempts: 6 [tags:chat-trace=4Qk-nlvC3Loh] 2019-10-15T10:10:02.211711Z ▶ [DEBU keybase connection.go:786] da75f (CONN gregor 37aef634) RetryNotify attempt 2019-10-15T10:10:02.211987Z ▶ [DEBU keybase connection.go:585] da760 (CONN gregor 37aef634) Connection: dialing transport 2019-10-15T10:10:02.212686Z ▶ [DEBU keybase connection.go:246] da761 (CONNTSPT 444c22b6) Dialing chat-0.core.keybaseapi.com:443 2019-10-15T10:10:02.227616Z ▶ [DEBU keybase connection.go:274] da762 (CONNTSPT 444c22b6) baseConn: 192.168.232.2:58531; Calling Handshake 2019-10-15T10:10:02.239161Z ▶ [WARN keybase connection.go:591] da763 (CONN gregor 37aef634) Connection: error dialing transport: x509: certificate signed by unknown authority 2019-10-15T10:10:02.239510Z ▶ [DEBU keybase gregor.go:899] da764 ++Chat: | PushHandler: should retry on connect, err x509: certificate signed by unknown authority 2019-10-15T10:10:02.239721Z ▶ [DEBU keybase connection.go:788] da765 (CONN gregor 37aef634) RetryNotify operation result: x509: certificate signed by unknown authority 2019-10-15T10:10:02.240001Z ▶ [DEBU keybase gregor.go:854] da766 ++Chat: + PushHandler: OnConnectError 2019-10-15T10:10:02.240368Z ▶ [DEBU keybase gregor.go:855] da767 ++Chat: | PushHandler: OnConnectError: err: x509: certificate signed by unknown authority, reconnect throttle duration: 2s 2019-10-15T10:10:02.240610Z ▶ [DEBU keybase gregor.go:866] da768 ++Chat: - PushHandler: OnConnectError -> ok [time=610.83µs] 2019-10-15T10:10:02.402547Z ▶ [DEBU kbfs connection.go:786] da769 (CONN MDServerRemote ec1ef460) RetryNotify attempt 2019-10-15T10:10:02.402809Z ▶ [DEBU kbfs connection.go:585] da76a (CONN MDServerRemote ec1ef460) Connection: dialing transport 2019-10-15T10:10:02.403478Z ▶ [DEBU kbfs connection.go:246] da76b (CONNTSPT 52cdb740) Dialing mdserver-1.kbfs.keybaseapi.com:443 2019-10-15T10:10:02.433540Z ▶ [DEBU kbfs connection.go:274] da76c (CONNTSPT 52cdb740) baseConn: 192.168.232.2:58533; Calling Handshake 2019-10-15T10:10:02.448570Z ▶ [WARN kbfs connection.go:591] da76d (CONN MDServerRemote ec1ef460) Connection: error dialing transport: x509: certificate signed by unknown authority 2019-10-15T10:10:02.452319Z ▶ [DEBU kbfs connection.go:788] da76e (CONN MDServerRemote ec1ef460) RetryNotify operation result: x509: certificate signed by unknown authority 2019-10-15T10:10:02.452902Z ▶ [WARN kbfs mdserver_remote.go:394] da76f MDServerRemote: connection error: "x509: certificate signed by unknown authority"; retrying in 2s 2019-10-15T10:10:02.454035Z ▶ [DEBU kbfs keybase_service_base.go:1138] da770 Sending notification for onlineStatus: online=false 2019-10-15T10:10:02.556252Z ▶ [DEBU kbfs(FBO 5a0d395c) folder_branch_ops.go:6194] da771 {VDL:1} Status [tags:SFSID=iuqnyWzifRfAcdr4631bUQ] ... 2019-10-15T10:10:03.489436Z ▶ [DEBU kbfs(BSR) connection.go:786] da78d (CONN BlockServerRemoteGet e897dfa8) RetryNotify attempt 2019-10-15T10:10:03.489780Z ▶ [DEBU kbfs(BSR) connection.go:585] da78e (CONN BlockServerRemoteGet e897dfa8) Connection: dialing transport 2019-10-15T10:10:03.490312Z ▶ [DEBU kbfs(BSR) connection.go:246] da78f (CONNTSPT e7520685) Dialing bserver-0.kbfs.keybaseapi.com:443 2019-10-15T10:10:03.508535Z ▶ [DEBU kbfs(BSR) connection.go:274] da790 (CONNTSPT e7520685) baseConn: 192.168.232.2:58534; Calling Handshake 2019-10-15T10:10:03.521691Z ▶ [WARN kbfs(BSR) connection.go:591] da791 (CONN BlockServerRemoteGet e897dfa8) Connection: error dialing transport: x509: certificate signed by unknown authority 2019-10-15T10:10:03.522866Z ▶ [DEBU kbfs(BSR) connection.go:788] da792 (CONN BlockServerRemoteGet e897dfa8) RetryNotify operation result: x509: certificate signed by unknown authority 2019-10-15T10:10:03.523094Z ▶ [WARN kbfs(BSR) bserver_remote.go:232] da793 BlockServerRemoteGet: connection error: x509: certificate signed by unknown authority; retrying in 2s 2019-10-15T10:10:03.694548Z ▶ [DEBU kbfs(BSR) connection.go:786] da794 (CONN BlockServerRemotePut 8bb1c24b) RetryNotify attempt 2019-10-15T10:10:03.695179Z ▶ [DEBU kbfs(BSR) connection.go:585] da795 (CONN BlockServerRemotePut 8bb1c24b) Connection: dialing transport 2019-10-15T10:10:03.695688Z ▶ [DEBU kbfs(BSR) connection.go:246] da796 (CONNTSPT 97c72001) Dialing bserver-1.kbfs.keybaseapi.com:443 2019-10-15T10:10:03.716027Z ▶ [DEBU kbfs(BSR) connection.go:274] da797 (CONNTSPT 97c72001) baseConn: 192.168.232.2:58535; Calling Handshake 2019-10-15T10:10:03.728109Z ▶ [WARN kbfs(BSR) connection.go:591] da798 (CONN BlockServerRemotePut 8bb1c24b) Connection: error dialing transport: x509: certificate signed by unknown authority 2019-10-15T10:10:03.731142Z ▶ [DEBU kbfs(BSR) connection.go:788] da799 (CONN BlockServerRemotePut 8bb1c24b) RetryNotify operation result: x509: certificate signed by unknown authority 2019-10-15T10:10:03.731344Z ▶ [WARN kbfs(BSR) bserver_remote.go:232] da79a BlockServerRemotePut: connection error: x509: certificate signed by unknown authority; retrying in 2s 2019-10-15T10:10:04.246345Z ▶ [DEBU keybase connection.go:786] da79b (CONN gregor 37aef634) RetryNotify attempt 2019-10-15T10:10:04.246623Z ▶ [DEBU keybase connection.go:585] da79c (CONN gregor 37aef634) Connection: dialing transport 2019-10-15T10:10:04.247438Z ▶ [DEBU keybase connection.go:246] da79d (CONNTSPT 444c22b6) Dialing chat-0.core.keybaseapi.com:443 2019-10-15T10:10:04.270784Z ▶ [DEBU keybase connection.go:274] da79e (CONNTSPT 444c22b6) baseConn: 192.168.232.2:58536; Calling Handshake 2019-10-15T10:10:04.279242Z ▶ [WARN keybase connection.go:591] da79f (CONN gregor 37aef634) Connection: error dialing transport: x509: certificate signed by unknown authority 2019-10-15T10:10:04.279531Z ▶ [DEBU keybase gregor.go:899] da7a0 ++Chat: | PushHandler: should retry on connect, err x509: certificate signed by unknown authority 2019-10-15T10:10:04.279738Z ▶ [DEBU keybase connection.go:788] da7a1 (CONN gregor 37aef634) RetryNotify operation result: x509: certificate signed by unknown authority 2019-10-15T10:10:04.280194Z ▶ [DEBU keybase gregor.go:854] da7a2 ++Chat: + PushHandler: OnConnectError 2019-10-15T10:10:04.280389Z ▶ [DEBU keybase gregor.go:855] da7a3 ++Chat: | PushHandler: OnConnectError: err: x509: certificate signed by unknown authority, reconnect throttle duration: 2s 2019-10-15T10:10:04.280677Z ▶ [DEBU keybase gregor.go:866] da7a4 ++Chat: - PushHandler: OnConnectError -> ok [time=481.67µs] 2019-10-15T10:10:04.458688Z ▶ [DEBU kbfs connection.go:786] da7a5 (CONN MDServerRemote ec1ef460) RetryNotify attempt 2019-10-15T10:10:04.463975Z ▶ [DEBU kbfs connection.go:585] da7a6 (CONN MDServerRemote ec1ef460) Connection: dialing transport 2019-10-15T10:10:04.464491Z ▶ [DEBU kbfs connection.go:246] da7a7 (CONNTSPT 52cdb740) Dialing mdserver-1.kbfs.keybase.io:443 2019-10-15T10:10:04.482311Z ▶ [DEBU kbfs connection.go:274] da7a8 (CONNTSPT 52cdb740) baseConn: 192.168.232.2:58538; Calling Handshake 2019-10-15T10:10:04.493255Z ▶ [WARN kbfs connection.go:591] da7a9 (CONN MDServerRemote ec1ef460) Connection: error dialing transport: x509: certificate signed by unknown authority 2019-10-15T10:10:04.493544Z ▶ [DEBU kbfs connection.go:788] da7aa (CONN MDServerRemote ec1ef460) RetryNotify operation result: x509: certificate signed by unknown authority 2019-10-15T10:10:04.493750Z ▶ [WARN kbfs mdserver_remote.go:394] da7ab MDServerRemote: connection error: "x509: certificate signed by unknown authority"; retrying in 2s 2019-10-15T10:10:04.497379Z ▶ [DEBU kbfs keybase_service_base.go:1138] da7ac Sending notification for onlineStatus: online=false 2019-10-15T10:10:05.524259Z ▶ [DEBU kbfs(BSR) connection.go:786] da7ad (CONN BlockServerRemoteGet e897dfa8) RetryNotify attempt 2019-10-15T10:10:05.524568Z ▶ [DEBU kbfs(BSR) connection.go:585] da7ae (CONN BlockServerRemoteGet e897dfa8) Connection: dialing transport 2019-10-15T10:10:05.525450Z ▶ [DEBU kbfs(BSR) connection.go:246] da7af (CONNTSPT e7520685) Dialing bserver-1.kbfs.keybase.io:443 2019-10-15T10:10:05.546579Z ▶ [DEBU kbfs(BSR) connection.go:274] da7b0 (CONNTSPT e7520685) baseConn: 192.168.232.2:58539; Calling Handshake 2019-10-15T10:10:05.556605Z ▶ [WARN kbfs(BSR) connection.go:591] da7b1 (CONN BlockServerRemoteGet e897dfa8) Connection: error dialing transport: x509: certificate signed by unknown authority 2019-10-15T10:10:05.556840Z ▶ [DEBU kbfs(BSR) connection.go:788] da7b2 (CONN BlockServerRemoteGet e897dfa8) RetryNotify operation result: x509: certificate signed by unknown authority 2019-10-15T10:10:05.557171Z ▶ [WARN kbfs(BSR) bserver_remote.go:232] da7b3 BlockServerRemoteGet: connection error: x509: certificate signed by unknown authority; retrying in 2s 2019-10-15T10:10:05.733733Z ▶ [DEBU kbfs(BSR) connection.go:786] da7b4 (CONN BlockServerRemotePut 8bb1c24b) RetryNotify attempt 2019-10-15T10:10:05.734006Z ▶ [DEBU kbfs(BSR) connection.go:585] da7b5 (CONN BlockServerRemotePut 8bb1c24b) Connection: dialing transport 2019-10-15T10:10:05.734501Z ▶ [DEBU kbfs(BSR) connection.go:246] da7b6 (CONNTSPT 97c72001) Dialing bserver-0.kbfs.keybase.io:443 2019-10-15T10:10:05.751253Z ▶ [DEBU kbfs(BSR) connection.go:274] da7b7 (CONNTSPT 97c72001) baseConn: 192.168.232.2:58540; Calling Handshake 2019-10-15T10:10:05.760768Z ▶ [WARN kbfs(BSR) connection.go:591] da7b8 (CONN BlockServerRemotePut 8bb1c24b) Connection: error dialing transport: x509: certificate signed by unknown authority 2019-10-15T10:10:05.761023Z ▶ [DEBU kbfs(BSR) connection.go:788] da7b9 (CONN BlockServerRemotePut 8bb1c24b) RetryNotify operation result: x509: certificate signed by unknown authority 2019-10-15T10:10:05.761243Z ▶ [WARN kbfs(BSR) bserver_remote.go:232] da7ba BlockServerRemotePut: connection error: x509: certificate signed by unknown authority; retrying in 2s 2019-10-15T10:10:06.282326Z ▶ [DEBU keybase connection.go:786] da7bb (CONN gregor 37aef634) RetryNotify attempt 2019-10-15T10:10:06.282862Z ▶ [DEBU keybase connection.go:585] da7bc (CONN gregor 37aef634) Connection: dialing transport 2019-10-15T10:10:06.283850Z ▶ [DEBU keybase connection.go:246] da7bd (CONNTSPT 444c22b6) Dialing chat-0.core.keybaseapi.com:443 2019-10-15T10:10:06.306601Z ▶ [DEBU keybase connection.go:274] da7be (CONNTSPT 444c22b6) baseConn: 192.168.232.2:58541; Calling Handshake 2019-10-15T10:10:06.320976Z ▶ [WARN keybase connection.go:591] da7bf (CONN gregor 37aef634) Connection: error dialing transport: x509: certificate signed by unknown authority 2019-10-15T10:10:06.330527Z ▶ [DEBU keybase gregor.go:899] da7c0 ++Chat: | PushHandler: should retry on connect, err x509: certificate signed by unknown authority 2019-10-15T10:10:06.332931Z ▶ [DEBU keybase connection.go:788] da7c1 (CONN gregor 37aef634) RetryNotify operation result: x509: certificate signed by unknown authority 2019-10-15T10:10:06.333420Z ▶ [DEBU keybase gregor.go:854] da7c2 ++Chat: + PushHandler: OnConnectError 2019-10-15T10:10:06.333614Z ▶ [DEBU keybase gregor.go:855] da7c3 ++Chat: | PushHandler: OnConnectError: err: x509: certificate signed by unknown authority, reconnect throttle duration: 2s 2019-10-15T10:10:06.333875Z ▶ [DEBU keybase gregor.go:866] da7c4 ++Chat: - PushHandler: OnConnectError -> ok [time=449.57µs] 2019-10-15T10:10:06.498176Z ▶ [DEBU kbfs connection.go:786] da7c5 (CONN MDServerRemote ec1ef460) RetryNotify attempt 2019-10-15T10:10:06.498451Z ▶ [DEBU kbfs connection.go:585] da7c6 (CONN MDServerRemote ec1ef460) Connection: dialing transport 2019-10-15T10:10:06.498940Z ▶ [DEBU kbfs connection.go:246] da7c7 (CONNTSPT 52cdb740) Dialing mdserver-0.kbfs.keybase.io:443 2019-10-15T10:10:06.517469Z ▶ [DEBU kbfs connection.go:274] da7c8 (CONNTSPT 52cdb740) baseConn: 192.168.232.2:58543; Calling Handshake 2019-10-15T10:10:06.526612Z ▶ [WARN kbfs connection.go:591] da7c9 (CONN MDServerRemote ec1ef460) Connection: error dialing transport: x509: certificate signed by unknown authority 2019-10-15T10:10:06.526889Z ▶ [DEBU kbfs connection.go:788] da7ca (CONN MDServerRemote ec1ef460) RetryNotify operation result: x509: certificate signed by unknown authority 2019-10-15T10:10:06.527191Z ▶ [WARN kbfs mdserver_remote.go:394] da7cb MDServerRemote: connection error: "x509: certificate signed by unknown authority"; retrying in 2s 2019-10-15T10:10:06.527625Z ▶ [DEBU kbfs keybase_service_base.go:1138] da7cc Sending notification for onlineStatus: online=false

[maxtaco]

Hard for us to debug this. Maybe your proxy is extra sketchy and is injecting its own certificate.

[ddworken]

@totonymous35 How are you adding the CA certificate to the OS trust store on Linux? I just tested it on Debian and the "standard" way of adding a new CA certificate doesn't seem to be fully compatible with how Go pulls the CA certificates out of the trust store. Can you make sure that /etc/ssl/certs/ca-certificates.crt contains your proxy CA certificate? Once I added it there, everything (kbfs, chat, etc) works correctly in my test VM.

See this for the source of that file and why it is required for it to be in there.