nextcloud / server

☁️ Nextcloud server, a safe home for all your data
https://nextcloud.com
GNU Affero General Public License v3.0
27.4k stars 4.07k forks source link

[Bug]: Server does not utilize system CA Certificates through specified HTTP Proxy #46621

Open tunloop opened 3 months ago

tunloop commented 3 months ago

⚠️ This issue respects the following points: ⚠️

Bug description

Nextcloud outbound HTTP communications through forward HTTP proxy specified in config.php does not utilize the system ca store (on Debian: /etc/ssl/certs/ca-certificates.crt) even when CA file is explicitly specified in both FPM and CLI php.ini configuration files. (cURL error 35 - unknown CA)

Steps to reproduce

  1. Specify HTTP proxy in config.php (ex. 'proxy' => '172.20.20.10:3142',)
  2. Ensure php.ini in both FPM and CLI have CA crt file specified (ex. openssl.cafile=/etc/ssl/certs/ca-certificates.crt)
  3. Test nextcloud's outbound connection by browsing to the app store in your web browser.
  4. See error (in nextcloud logs and squid logs)

Expected behavior

Nextcloud, when using an HTTP proxy, should trust all CA certificates in system store.

Installation method

N/A

Nextcloud Server version

28

Operating system

Debian 12

PHP engine version

PHP 8.3

Web server

NGINX

Database engine version

MariaDB

Is this bug present after an update or on a fresh install?

Unk/N/A

Are you using the Nextcloud Server Encryption module?

No

What user-backends are you using?

Configuration report

{
    "system": {
        "proxy": "apt.example.com:3142",
        "proxyexclude": [
            "nextcloud.example.com"
        ],
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "172.20.20.5",
            "nextcloud.example.com",
            "172.20.20.6"
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "mysql",
        "version": "29.0.3.4",
        "overwrite.cli.url": "http:\/\/nextcloud.example.com\/nextcloud",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbtableprefix": "oc_",
        "mysql.utf8mb4": true,
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "maintenance": false,
        "maintenance_window_start": 1,
        "theme": "",
        "loglevel": 2,
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 0,
            "dbindex": 0,
            "timeout": 1.5
        },
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "default_phone_region": "US",
        "allow_local_remote_servers": true,
        "mail_smtpmode": "sendmail",
        "mail_sendmailmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "app_install_overwrite": [
            "breezedark"
        ],
        "app.mail.verify-tls-peer": false,
        "memories.exiftool_no_local": true,
        "memories.vod.path": "\/var\/www\/nextcloud\/apps\/memories\/bin-ext\/go-vod-amd64"
    }
}

List of activated Apps

Enabled:
  - admin_audit: 1.19.0
  - bruteforcesettings: 2.9.0
  - calendar: 4.7.12
  - checksum: 1.2.4
  - circles: 29.0.0-dev
  - cloud_federation_api: 1.12.0
  - collectives: 2.12.0
  - comments: 1.19.0
  - contactsinteraction: 1.10.0
  - dashboard: 7.9.0
  - dav: 1.30.1
  - external: 5.4.0
  - federatedfilesharing: 1.19.0
  - files: 2.1.0
  - files_downloadlimit: 2.0.0
  - files_pdfviewer: 2.10.0
  - files_reminders: 1.2.0
  - files_sharing: 1.21.0
  - files_trashbin: 1.19.0
  - files_versions: 1.22.0
  - logreader: 2.14.0
  - lookup_server_connector: 1.17.0
  - mail: 3.7.5
  - maps: 1.4.0
  - notes: 4.10.0
  - notifications: 2.17.0
  - oauth2: 1.17.0
  - password_policy: 1.19.0
  - photos: 2.5.0
  - privacy: 1.13.0
  - provisioning_api: 1.19.0
  - related_resources: 1.4.0
  - richdocuments: 8.4.3
  - riotchat: 0.17.6
  - serverinfo: 1.19.0
  - settings: 1.12.0
  - suspicious_login: 7.0.0
  - systemtags: 1.19.0
  - tables: 0.7.4
  - text: 3.10.1
  - theming: 2.4.0
  - theming_customcss: 1.16.0
  - twofactor_backupcodes: 1.18.0
  - updatenotification: 1.19.1
  - user_status: 1.9.0
  - viewer: 2.3.0
  - weather_status: 1.9.0
  - workflowengine: 2.11.0
Disabled:
  - activity: 2.21.1 (installed 2.13.4)
  - encryption: 2.17.0
  - federation: 1.19.0 (installed 1.10.1)
  - files_archive: 1.2.3 (installed 1.2.3)
  - files_external: 1.21.0
  - firstrunwizard: 2.18.0 (installed 2.9.0)
  - nextcloud_announcements: 1.18.0 (installed 1.9.0)
  - recommendations: 2.1.0 (installed 0.8.0)
  - sharebymail: 1.19.0 (installed 1.10.0)
  - support: 1.12.0 (installed 1.3.0)
  - survey_client: 1.17.0 (installed 1.8.0)
  - twofactor_totp: 11.0.0-dev
  - user_ldap: 1.20.0

Nextcloud Signing status

No errors have been found.

Nextcloud Logs - Limited (Github 65K character limit)

{"reqId":"LhQTAEFZsygTyEd2JOSG","level":2,"time":"2024-07-18T19:26:46+00:00","remoteAddr":"10.10.0.2","user":"bigdog","app":"appstoreFetcher","method":"GET","url":"/settings/apps","message":"Could not connect to appstore: cURL error 35: OpenSSL/3.0.13: error:8000000D:system library::Permission denied (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://apps.nextcloud.com/api/v1/apps.json","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:128.0) Gecko/20100101 Firefox/128.0","version":"29.0.3.4","data":{"app":"appstoreFetcher"}}
{"reqId":"aptq5AybKgsT7H77SUWE","level":3,"time":"2024-07-18T19:32:28+00:00","remoteAddr":"10.10.0.2","user":"bigdog","app":"internet_connection_check","method":"GET","url":"/settings/ajax/checksetup","message":"Cannot connect to: www.nextcloud.com","userAgent":"Mozilla/5.0 (X11; Linux x86_64; rv:128.0) Gecko/20100101 Firefox/128.0","version":"29.0.3.4","exception":{"Exception":"GuzzleHttp\\Exception\\ConnectException","Message":"cURL error 35: OpenSSL/3.0.13: error:8000000D:system library::Permission denied (see https://curl.haxx.se/libcurl/c/libcurl-errors.html) for https://nextcloud.com/","Code":0,"Trace":[{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/CurlFactory.php","line":158,"function":"createRejection","class":"GuzzleHttp\\Handler\\CurlFactory","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/CurlFactory.php","line":110,"function":"finishError","class":"GuzzleHttp\\Handler\\CurlFactory","type":"::"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/CurlHandler.php","line":47,"function":"finish","class":"GuzzleHttp\\Handler\\CurlFactory","type":"::"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Middleware.php","line":142,"function":"__invoke","class":"GuzzleHttp\\Handler\\CurlHandler","type":"->"},{"file":"/var/www/nextcloud/lib/private/Http/Client/DnsPinMiddleware.php","line":123,"function":"GuzzleHttp\\{closure}","class":"GuzzleHttp\\Middleware","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/PrepareBodyMiddleware.php","line":35,"function":"OC\\Http\\Client\\{closure}","class":"OC\\Http\\Client\\DnsPinMiddleware","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Middleware.php","line":31,"function":"__invoke","class":"GuzzleHttp\\PrepareBodyMiddleware","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/RedirectMiddleware.php","line":71,"function":"GuzzleHttp\\{closure}","class":"GuzzleHttp\\Middleware","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/RedirectMiddleware.php","line":107,"function":"__invoke","class":"GuzzleHttp\\RedirectMiddleware","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/RedirectMiddleware.php","line":73,"function":"checkRedirect","class":"GuzzleHttp\\RedirectMiddleware","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/FulfilledPromise.php","line":41,"function":"GuzzleHttp\\{closure}","class":"GuzzleHttp\\RedirectMiddleware","type":"->","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/TaskQueue.php","line":48,"function":"GuzzleHttp\\Promise\\{closure}","class":"GuzzleHttp\\Promise\\FulfilledPromise","type":"::","args":["*** sensitive parameters replaced ***"]},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":248,"function":"run","class":"GuzzleHttp\\Promise\\TaskQueue","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":224,"function":"invokeWaitFn","class":"GuzzleHttp\\Promise\\Promise","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":269,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":226,"function":"invokeWaitList","class":"GuzzleHttp\\Promise\\Promise","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/promises/src/Promise.php","line":62,"function":"waitIfPending","class":"GuzzleHttp\\Promise\\Promise","type":"->"},{"file":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Client.php","line":189,"function":"wait","class":"GuzzleHttp\\Promise\\Promise","type":"->"},{"file":"/var/www/nextcloud/lib/private/Http/Client/Client.php","line":230,"function":"request","class":"GuzzleHttp\\Client","type":"->"},{"file":"/var/www/nextcloud/apps/settings/lib/SetupChecks/InternetConnectivity.php","line":85,"function":"get","class":"OC\\Http\\Client\\Client","type":"->"},{"file":"/var/www/nextcloud/apps/settings/lib/SetupChecks/InternetConnectivity.php","line":68,"function":"isSiteReachable","class":"OCA\\Settings\\SetupChecks\\InternetConnectivity","type":"->"},{"file":"/var/www/nextcloud/lib/private/SetupCheck/SetupCheckManager.php","line":51,"function":"run","class":"OCA\\Settings\\SetupChecks\\InternetConnectivity","type":"->"},{"file":"/var/www/nextcloud/apps/settings/lib/Controller/CheckSetupController.php","line":179,"function":"runAll","class":"OC\\SetupCheck\\SetupCheckManager","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":232,"function":"check","class":"OCA\\Settings\\Controller\\CheckSetupController","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/Http/Dispatcher.php","line":138,"function":"executeController","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/AppFramework/App.php","line":184,"function":"dispatch","class":"OC\\AppFramework\\Http\\Dispatcher","type":"->"},{"file":"/var/www/nextcloud/lib/private/Route/Router.php","line":338,"function":"main","class":"OC\\AppFramework\\App","type":"::"},{"file":"/var/www/nextcloud/lib/base.php","line":1050,"function":"match","class":"OC\\Route\\Router","type":"->"},{"file":"/var/www/nextcloud/index.php","line":49,"function":"handleRequest","class":"OC","type":"::"}],"File":"/var/www/nextcloud/3rdparty/guzzlehttp/guzzle/src/Handler/CurlFactory.php","Line":210,"message":"Cannot connect to: www.nextcloud.com","exception":{},"CustomMessage":"Cannot connect to: www.nextcloud.com"}}

Additional info

No response

tunloop commented 3 months ago

Additional information, since post was limited by characters

Jul 18 12:57:03 WebProxy squid[108450]: ERROR: failure while accepting a TLS connection on conn67 local=172.20.20.10:3142 remote=172.20.20.5:48320 FD 13 flags=1: 0x5a66a2522130*1
                                            current master transaction: master89
Jul 18 12:57:03 WebProxy squid[108450]: {"log":"squid-access","time":"18/Jul/2024:12:57:03 -0700","responseTime":21,"srcIp":"172.20.20.5","srcPort":"48320","destIp":"-","destPort":"-","userIdent":"-","user":"-","method":"CONNECT","httpVer":"1.1","url":"apps.nextcloud.com:443","referrer":"-","userAgent":"-","status":"200","reqAction":"NONE_NONE","reqStatus":"HIER_NONE","contentType":"-","bytes":103,"bytesIn":103,"bytesOut":0}

PCAP showing that nextcloud is closing the connection because it does not trust the CA cert. caunknown

Not an issue with Squid, curl on the nextcloud server using the system CA cert store functions just fine:


curl --cacert /etc/ssl/certs/ca-certificates.crt -vvv https://apps.nextcloud.com > /tmp/file
* Uses proxy env variable https_proxy == 'http://apt.example.com:3142'
Trying 172.20.20.10:3142...
* Connected to apt.example.com (172.20.20.10) port 3142 (#0)
* allocate connect buffer
* Establish HTTP proxy tunnel to apps.nextcloud.com:443
> CONNECT apps.nextcloud.com:443 HTTP/1.1
> Host: apps.nextcloud.com:443
> User-Agent: curl/7.88.1
> Proxy-Connection: Keep-Alive
> 
< HTTP/1.1 200 Connection established
< 
* CONNECT phase completed
* CONNECT tunnel established, response 200
* ALPN: offers h2,http/1.1
} [5 bytes data]
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
} [512 bytes data]
*  CAfile: /etc/ssl/certs/ca-certificates.crt
*  CApath: /etc/ssl/certs
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Server hello (2):
{ [122 bytes data]
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
{ [6 bytes data]
* TLSv1.3 (IN), TLS handshake, Certificate (11):
{ [2918 bytes data]
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
{ [520 bytes data]
* TLSv1.3 (IN), TLS handshake, Finished (20):
{ [52 bytes data]
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
} [1 bytes data]
* TLSv1.3 (OUT), TLS handshake, Finished (20):
} [52 bytes data]
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN: server did not agree on a protocol. Uses default.
* Server certificate:
*  subject: CN=apps.nextcloud.com
*  start date: Jul 13 23:41:26 2024 GMT
*  expire date: Jul 11 23:41:26 2034 GMT
*  subjectAltName: host "apps.nextcloud.com" matched cert's "apps.nextcloud.com"
*  issuer: C=US; ST=Washington; O=; CN=example.com; emailAddress=admin@example.com
*  SSL certificate verify ok.
* using HTTP/1.x
} [5 bytes data]
> GET / HTTP/1.1
> Host: apps.nextcloud.com
> User-Agent: curl/7.88.1
> Accept: */*
> 
{ [5 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [233 bytes data]
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
{ [233 bytes data]
* old SSL session ID is stale, removing
< HTTP/1.1 200 OK
joshtrichards commented 3 months ago

Related: nextcloud/documentation#11906

tunloop commented 3 months ago

Is the fix to add the certificate to the occ cert store? sudo -u www-data php occ security:certificates:import /path/to/certificate

Is the related issue about just improving documentation, or making nextcloud utilize system certificate stores?