nextcloud / server

☁️ Nextcloud server, a safe home for all your data
https://nextcloud.com
GNU Affero General Public License v3.0
27.53k stars 4.08k forks source link

[Bug]: NC29 .well-known URLs, failed on: /.well-known/caldav #45033

Closed cvandesande closed 2 months ago

cvandesande commented 7 months ago

⚠️ This issue respects the following points: ⚠️

Bug description

With NC28 I had no well-known URL errors, and no change to the NGINX configuration. After upgrading to NC29, I now have the following message: Your web server is not properly set up to resolve .well-known URLs, failed on: /.well-known/caldav For more details see the [documentation ↗](https://docs.nextcloud.com/server/29/go.php?to=admin-setup-well-known-URL).

In the NGINX logs, I see a 401 errors:

192.168.9.6 - - [25/Apr/2024:13:02:21 +0000] "PROPFIND /.well-known/caldav HTTP/1.1" 301 162 "-" "Nextcloud Server Crawler"
192.168.9.6 - - [25/Apr/2024:13:02:21 +0000] "GET /remote.php/dav HTTP/1.1" 401 569 "-" "Nextcloud Server Crawler"
192.168.9.6 - - [25/Apr/2024:13:02:21 +0000] "GET /.well-known/caldav HTTP/1.1" 301 162 "-" "Nextcloud Server Crawler"
192.168.9.6 - - [25/Apr/2024:13:02:21 +0000] "GET /remote.php/dav HTTP/1.1" 401 569 "-" "Nextcloud Server Crawler"

My Android DAVx5 client doesn't seem to have any issues and continues to work well.

A curl test, shows the 301 redirect working, followed by a 401, but I'm assume that's expected with an unauthenticated request

curl -IL https://nextcloud.mydomain.com/.well-known/caldav

HTTP/2 301 
server: nginx/1.25.4
date: Thu, 25 Apr 2024 13:12:51 GMT
content-type: text/html
content-length: 162
location: https://nextcloud.mydomain.com/remote.php/dav
referrer-policy: no-referrer
x-content-type-options: nosniff
x-download-options: noopen
x-frame-options: SAMEORIGIN
x-permitted-cross-domain-policies: none
x-robots-tag: noindex, nofollow
x-xss-protection: 1; mode=block
strict-transport-security: max-age=15768000; includeSubDomains; preload;

HTTP/2 401 
server: nginx/1.25.4
date: Thu, 25 Apr 2024 13:12:51 GMT
content-type: application/xml; charset=utf-8
set-cookie: oc_sessionPassphrase=NoBfvCYLv%2B7Hzw7eRoiL4VWZNyADXzcx2k5fgbu4FerYYBJjgl%2Fq8xLrTWscFxu3ithDlInkdabcfGg0nbL0wrG3B%2BPRkOCYjtibS4QvsluoWlkgrT5DdSWiQGUwHxi9; path=/; secure; HttpOnly; SameSite=Lax
set-cookie: __Host-nc_sameSiteCookielax=true; path=/; httponly;secure; expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=lax
set-cookie: __Host-nc_sameSiteCookiestrict=true; path=/; httponly;secure; expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=strict
set-cookie: oc2a3d1f7bcc=6eadda8dbfe0995012c16997240ee6b0; path=/; secure; HttpOnly; SameSite=Lax
expires: Thu, 19 Nov 1981 08:52:00 GMT
cache-control: no-store, no-cache, must-revalidate
pragma: no-cache
content-security-policy: default-src 'none';
www-authenticate: Basic realm="MyDomain, charset="UTF-8"
referrer-policy: no-referrer
x-content-type-options: nosniff
x-download-options: noopen
x-frame-options: SAMEORIGIN
x-permitted-cross-domain-policies: none
x-robots-tag: noindex, nofollow
x-xss-protection: 1; mode=block
strict-transport-security: max-age=15768000; includeSubDomains; preload;

There is an error in Nextcloud.log that appears relevant:

    "File": "/usr/share/nginx/html/nextcloud/3rdparty/sabre/dav/lib/DAV/Auth/Plugin.php",
    "Line": 152,
    "message": "No public access to this resource., No 'Authorization: Basic' header found. Either the client didn't send one, or the server is misconfigured, No 'Authorization: Bearer' header found. Either the client didn't send one, or the server is mis-configured, No 'Authorization: Basic' header found. Either the client didn't send one, or the server is misconfigured"

Steps to reproduce

  1. Upgrade from NC28 to NC29
  2. well-known message in admin settings

Expected behavior

No well-known errors

Installation method

Community Manual installation with Archive

Nextcloud Server version

29

Operating system

Other

PHP engine version

PHP 8.2

Web server

Nginx

Database engine version

PostgreSQL

Is this bug present after an update or on a fresh install?

Upgraded to a MAJOR version (ex. 22 to 23)

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

Configuration report

{
    "system": {
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "nextcloud.mydomain.com",
            "nextcloud"
        ],
        "apps_paths": [
            {
                "path": "\/usr\/share\/nginx\/html\/nextcloud\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/usr\/share\/nginx\/html\/nextcloud\/custom-apps",
                "url": "\/custom-apps",
                "writable": true
            }
        ],
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "pgsql",
        "version": "29.0.0.19",
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "forwarded_for_headers": [
            "HTTP_X_FORWARDED_FOR",
            "HTTP_X_FORWARDED",
            "HTTP_FORWARDED_FOR"
        ],
        "overwriteprotocol": "https",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbtableprefix": "oc_",
        "installed": true,
        "mail_smtpmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "465",
        "loglevel": 0,
        "maintenance": false,
        "enable_previews": true,
        "secret": "***REMOVED SENSITIVE VALUE***",
        "filesystem_check_changes": 0,
        "filelocking.enabled": "true",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": 6379,
            "timeout": 1.5,
            "read_timeout": 1.5,
            "dbindex": 0
        },
        "trashbin_retention_obligation": "auto",
        "overwrite.cli.url": "https:\/\/nextcloud.mydomain.com",
        "mail_smtpauthtype": "LOGIN",
        "mail_smtpsecure": "ssl",
        "theme": "",
        "app_install_overwrite": [
            "joplin"
        ],
        "encryption.legacy_format_support": false,
        "encryption.key_storage_migrated": false,
        "default_language": "en",
        "default_phone_region": "CA",
        "maintenance_window_start": 1,
        "memories.db.triggers.fcu": true,
        "memories.exiftool": "\/usr\/share\/nginx\/html\/nextcloud\/custom-apps\/memories\/bin-ext\/exiftool-amd64-glibc",
        "memories.vod.path": "\/usr\/share\/nginx\/html\/nextcloud\/custom-apps\/memories\/bin-ext\/go-vod-amd64",
        "memories.gis_type": 2,
        "enabledPreviewProviders": [
            "OC\\Preview\\Image",
            "OC\\Preview\\HEIC",
            "OC\\Preview\\Movie",
            "OC\\Preview\\TIFF"
        ]
    }
}

List of activated Apps

No response

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

{
  "reqId": "6jDGw0WbOAAarIrEpjXS",
  "level": 0,
  "time": "2024-04-25T13:21:06+00:00",
  "remoteAddr": "192.168.9.6",
  "user": "--",
  "app": "webdav",
  "method": "GET",
  "url": "/remote.php/dav",
  "message": "No public access to this resource., No 'Authorization: Basic' header found. Either the client didn't send one, or the server is misconfigured, No 'Authorization: Bearer' header found. Either the client didn't send one, or the server is mis-configured, No 'Authorization: Basic' header found. Either the client didn't send one, or the server is misconfigured",
  "userAgent": "Nextcloud Server Crawler",
  "version": "29.0.0.19",
  "exception": {
    "Exception": "Sabre\\DAV\\Exception\\NotAuthenticated",
    "Message": "No public access to this resource., No 'Authorization: Basic' header found. Either the client didn't send one, or the server is misconfigured, No 'Authorization: Bearer' header found. Either the client didn't send one, or the server is mis-configured, No 'Authorization: Basic' header found. Either the client didn't send one, or the server is misconfigured",
    "Code": 0,
    "Trace": [
      {
        "file": "/usr/share/nginx/html/nextcloud/3rdparty/sabre/event/lib/WildcardEmitterTrait.php",
        "line": 89,
        "function": "beforeMethod",
        "class": "Sabre\\DAV\\Auth\\Plugin",
        "type": "->",
        "args": [
          [
            "Sabre\\HTTP\\Request"
          ],
          [
            "Sabre\\HTTP\\Response"
          ]
        ]
      },
      {
        "file": "/usr/share/nginx/html/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
        "line": 456,
        "function": "emit",
        "class": "Sabre\\DAV\\Server",
        "type": "->",
        "args": [
          "beforeMethod:GET",
          [
            [
              "Sabre\\HTTP\\Request"
            ],
            [
              "Sabre\\HTTP\\Response"
            ]
          ]
        ]
      },
      {
        "file": "/usr/share/nginx/html/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
        "line": 253,
        "function": "invokeMethod",
        "class": "Sabre\\DAV\\Server",
        "type": "->",
        "args": [
          [
            "Sabre\\HTTP\\Request"
          ],
          [
            "Sabre\\HTTP\\Response"
          ]
        ]
      },
      {
        "file": "/usr/share/nginx/html/nextcloud/3rdparty/sabre/dav/lib/DAV/Server.php",
        "line": 321,
        "function": "start",
        "class": "Sabre\\DAV\\Server",
        "type": "->",
        "args": []
      },
      {
        "file": "/usr/share/nginx/html/nextcloud/apps/dav/lib/Server.php",
        "line": 374,
        "function": "exec",
        "class": "Sabre\\DAV\\Server",
        "type": "->",
        "args": []
      },
      {
        "file": "/usr/share/nginx/html/nextcloud/apps/dav/appinfo/v2/remote.php",
        "line": 35,
        "function": "exec",
        "class": "OCA\\DAV\\Server",
        "type": "->",
        "args": []
      },
      {
        "file": "/usr/share/nginx/html/nextcloud/remote.php",
        "line": 172,
        "args": [
          "/usr/share/nginx/html/nextcloud/apps/dav/appinfo/v2/remote.php"
        ],
        "function": "require_once"
      }
    ],
    "File": "/usr/share/nginx/html/nextcloud/3rdparty/sabre/dav/lib/DAV/Auth/Plugin.php",
    "Line": 152,
    "message": "No public access to this resource., No 'Authorization: Basic' header found. Either the client didn't send one, or the server is misconfigured, No 'Authorization: Bearer' header found. Either the client didn't send one, or the server is mis-configured, No 'Authorization: Basic' header found. Either the client didn't send one, or the server is misconfigured",
    "exception": {},
    "CustomMessage": "No public access to this resource., No 'Authorization: Basic' header found. Either the client didn't send one, or the server is misconfigured, No 'Authorization: Bearer' header found. Either the client didn't send one, or the server is mis-configured, No 'Authorization: Basic' header found. Either the client didn't send one, or the server is misconfigured"
  }
}

Additional info

No response

meldarionqeusse commented 7 months ago

I have the same problem and noticed my .htaccess file was changed with the update are you also getting a warning that .htaccess is not working?

cvandesande commented 7 months ago

@meldarionqeusse I have the .htaccess file but as I'm using NGINX for the web server I think NGINX ignores it?

I think the well-known URLs are working. I created a new app password and passed the credentials with curl, the redirect works and I get a valid response. So I think the well-known redirects are working, but something has gone wrong with the test in admin settings and no authentication is passed.

curl -iL --location-trusted -u validuser:validpassword https://nextcloud.mydomain.com/.well-known/caldav

HTTP/2 301 
server: nginx/1.25.4
date: Thu, 25 Apr 2024 13:55:24 GMT
content-type: text/html
content-length: 162
location: https://nextcloud.mydomain.com/remote.php/dav
referrer-policy: no-referrer
x-content-type-options: nosniff
x-download-options: noopen
x-frame-options: SAMEORIGIN
x-permitted-cross-domain-policies: none
x-robots-tag: noindex, nofollow
x-xss-protection: 1; mode=block
strict-transport-security: max-age=15768000; includeSubDomains; preload;

HTTP/2 200 
server: nginx/1.25.4
date: Thu, 25 Apr 2024 13:55:25 GMT
content-type: text/html; charset=UTF-8
set-cookie: oc_sessionPassphrase=BYDq5MblWFD0ccawoP0IeYzmlmUteNvFJ%2FaUKJIEZhRdAc4hkrKkA9NPlui%2FaI9QfbW%2F1MmckdtmPRfrlg4oIm8XeaMXLECM0NqBOwdhhf1oeQyeNd5ixn7h%2FkkIY5QA; path=/; secure; HttpOnly; SameSite=Lax
set-cookie: __Host-nc_sameSiteCookielax=true; path=/; httponly;secure; expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=lax
set-cookie: __Host-nc_sameSiteCookiestrict=true; path=/; httponly;secure; expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=strict
content-security-policy: default-src 'none';
expires: Thu, 19 Nov 1981 08:52:00 GMT
cache-control: no-store, no-cache, must-revalidate
pragma: no-cache
set-cookie: oc2a3d1f7bcc=7c4296af03227b5fa1a1eb6b2036acf1; path=/; secure; HttpOnly; SameSite=Lax
x-request-id: W9r66ZwhVZCA74Qr6dkT
x-debug-token: W9r66ZwhVZCA74Qr6dkT
referrer-policy: no-referrer
x-content-type-options: nosniff
x-download-options: noopen
x-frame-options: SAMEORIGIN
x-permitted-cross-domain-policies: none
x-robots-tag: noindex, nofollow
x-xss-protection: 1; mode=block
strict-transport-security: max-age=15768000; includeSubDomains; preload;

This is the WebDAV interface. It can only be accessed by WebDAV clients such as the Nextcloud desktop sync client
H5N1v2 commented 7 months ago

I have the same problem after upgrading to NC29, using apache2, php8.2-fpm and nginx proxy manager. But everything seems to be working fine for now.

ne20002 commented 7 months ago

I have this with 28.0.4. Community Docker image (fpm).

kesselb commented 7 months ago

Starting with 29 the check is done via php and no longer by your browser. We are using the overwrite.cli.url for it.

Here is the code: https://github.com/nextcloud/server/blob/master/apps/settings/lib/SetupChecks/WellKnownUrls.php

To debug it further please check the redirect on the server with curl and use the overwrite url.

cvandesande commented 7 months ago

@kesselb overwrite URL is the public facing domain

grep overwrite config.php 
  'overwriteprotocol' => 'https',
  'overwrite.cli.url' => 'https://nextcloud.mydomain.com',
  'app_install_overwrite' => 

It's the same URL I used in my successful curl test should it be something different?

GrahamTolhurst commented 7 months ago

I'm getting the same problem after upgrading to NC29.0.0. The error must be inaccurately reported as both my calendar and contacts are redirecting properly and syncing to my mobile device. A bit of testing: ./nextcloud/apps/settings/lib/SetupChecks/WellKnownUrls.php has the following lines of code:

['propfind', '/.well-known/carddav', [207], false],
['propfind', '/.well-known/caldav', [207], false],

With the above lines of code enabled, Nextcloud reports a "/.well-known/caldav" error. If I comment out the carddav line, Nextcloud reports the same caldav error. But if I comment out the caldav line and uncomment the carddav line, Nextcloud now reports a carddav error.

It's looking like it's reporting the last test as an error even though it's not.

If I comment out both lines, there's no error reported.

(NC29.0.0 with Apache2, behind Reverse Proxy (Nginx) Server)

akn01 commented 7 months ago

Same problem here. I use 'overwrite.cli.url' => 'https://nextcloud.mydomain.com', and curl -L -u validuser:validpassword https://nextcloud.mydomain.com/.well-known/caldav outputs the WebDAV message of the interface correctly.

grogg commented 7 months ago

Same problem after upgrade to Nextcloud 29.0.0 on Apache2. The install documentation has the redirects set to "301" in .htaccess and the error occurs even though the redirects have been verified as functioning properly. Changing the .htaccess redirects from "301" to "207" has resolved the errors on my end.

kesselb commented 7 months ago

Changing the .htaccess redirects from "301" to "207" has resolved the errors on my end.

Does the redirect still work? ;)

We are looking into the problem and if you are certain that the redirects are okay, then please ignore the setup check for now. Please don't change the response code to 207, that's just wrong.

kesselb commented 7 months ago

@cvandesande

The setup check expects a redirect to /remote.php/dav/.^1

Your redirect goes to /remote.php/dav without a trailing slash.

Please double-check your configuration if the redirects are configured with a trailing slash.^2

KopfKrieg commented 7 months ago

Seems like the documentation isn't really clear about that :/

For apache, there are no trailing slashes mentioned: https://docs.nextcloud.com/server/latest/admin_manual/issues/general_troubleshooting.html#service-discovery-label

Also, for reverse proxies there are some with, some without trailing slashes: https://docs.nextcloud.com/server/29/admin_manual/configuration_server/reverse_proxy_configuration.html#service-discovery

(Also, my Gnome Online Account [Fedora 40, Gnome 46] still doesn't work after adding trailing slashes, DAVx5 works fine though, and the error in the web ui is also gone)

kesselb commented 7 months ago

I think for the functionality - the service discovery - the trailing slash is not that important. The check, that works different since 29 than before, is rather strict and tests for the trailing slash.

In any case, the documentation should be updated. The service discovery is usually a topic if you are using a reverse proxy (without it should work out of the box) and therefore it's a part of that documentation. For the other cases, we have the article in the general troubleshooting section. It should be fine to move the service discovery to an own page in configuration_server and merge the articles.

akn01 commented 7 months ago

Solution for me: Redirection to /remote.php/dav/ with trailing slash at the end in Nginx Proxy Manager makes the error message disappear.

cvandesande commented 7 months ago

Trailing slash was the solution for me as well. Thank you! Agree with @KopfKrieg trailing slashes are missing from the NGINX reverse proxy docs: https://docs.nextcloud.com/server/latest/admin_manual/configuration_server/reverse_proxy_configuration.html#nginx

@kesselb should we keep this issue open, or close it and open a documentation issue?

janusn commented 7 months ago

Unfortunately, a trailing slash does not help my nc 29 instance. It still complain with either the following lines:

    location ^~ /.well-known {
        # The rules in this block are an adaptation of the rules
        # in `.htaccess` that concern `/.well-known`.

        location = /.well-known/carddav { return 301 $scheme://$host/remote.php/dav/; }
        location = /.well-known/caldav  { return 301 $scheme://$host/remote.php/dav/; }
    }

or

    location ^~ /.well-known {
        # The rules in this block are an adaptation of the rules
        # in `.htaccess` that concern `/.well-known`.

        rewrite ^/\.well-known/carddav https://$server_name/remote.php/dav/ redirect;
        rewrite ^/\.well-known/caldav https://$server_name/remote.php/dav/ redirect;
    }
blaine07 commented 7 months ago

Came here to confirm, changing THIS: orig-cal-nc29

To THIS: image

Fixed THIS issue when upgrading to v29.0.0: image png 183e5a6dc8a50c2c2694ffd3ef6cf707

I also restarted Nextcloud instance and my NPM Proxy it was going through :-)

sn0n commented 7 months ago

having this issue as well.

Caddy setup as reverse proxy, nextcloud manual install on its own VM. Caddy rewrite is working, returning a 401 result, resulting warning in Admin Overview.

h9419 commented 7 months ago

./nextcloud/apps/settings/lib/SetupChecks/WellKnownUrls.php has the following lines of code:

['propfind', '/.well-known/carddav', [207], false],
['propfind', '/.well-known/caldav', [207], false],

(NC29.0.0 with Apache2, behind Reverse Proxy (Nginx) Server)

This causes problem for subfolder installations because this will be checking for /subfolder/.well-known/carddav instead of /.well-known/carddav whereas applications using WebDAV will be querying for /.well-known/carddav without the subfolder.

I have a workaround for nginx/swag in https://github.com/linuxserver/reverse-proxy-confs/pull/673 if anyone is facing the same problem as I do

kesselb commented 7 months ago

@h9419

This causes problem for subfolder installations because this will be checking for /subfolder/.well-known/carddav instead of /.well-known/carddav whereas applications using WebDAV will be querying for /.well-known/carddav without the subfolder.

If overwrite.cli.url is yourhost.com/subfolder, then it will indeed check for yourhost.com/subfolder/.well-known/carddav.

The check also takes the trusted_domains into account, which should be declared without a subfolder and make the check pass.

Mind to take a look at your trusted_domains and share them with us?

sn0n commented 7 months ago

My trusted domains array cloud.domain.com, overwritecli has the same domain with https://

Not using a subfolder.

kesselb commented 7 months ago

My trusted domains array cloud.domain.com, overwritecli has the same domain with https://

Did you try to update the caddy redirect rules so they redirect to /remote.php/dav/ rather than /remote.php/dav (note the additional trailing slash)?

sn0n commented 6 months ago

Did you try to update the caddy redirect rules so they redirect to /remote.php/dav/ rather than /remote.php/dav (note the additional trailing slash)?

Yes, that wasn't the fix.

cruzmanny commented 6 months ago

I struggled with this issue for the past 48 hours. I am using Nextcloud 29. At first I thought it was an issue with upgrading to Ubuntu Server 24.04. However, adding 'localhost' to config/config.php file fixed it for me. I updated my file to this;

'trusted_domains' => array ( 0 => 'nextcloud.mydomain.com', 1 => 'localhost' ),

I hope this helps any one else who is struggling.

sn0n commented 6 months ago

I struggled with this issue for the past 48 hours. I am using Nextcloud 29. At first I thought it was an issue with upgrading to Ubuntu Server 24.04. However, adding 'localhost' to config/config.php file fixed it for me. I updated my file to this;

'trusted_domains' => array ( 0 => 'nextcloud.mydomain.com', 1 => 'localhost' ),

I hope this helps any one else who is struggling.

Sadly didn't work on my end either. With or without the trailing / on the redirect.

cruzmanny commented 6 months ago

I struggled with this issue for the past 48 hours. I am using Nextcloud 29. At first I thought it was an issue with upgrading to Ubuntu Server 24.04. However, adding 'localhost' to config/config.php file fixed it for me. I updated my file to this; 'trusted_domains' => array ( 0 => 'nextcloud.mydomain.com', 1 => 'localhost' ), I hope this helps any one else who is struggling.

Sadly didn't work on my end either. With or without the trailing / on the redirect.

The other thing I added when it started working was 'htaccess.RewriteBase' => '/', but I am not sure if that will work.

sn0n commented 6 months ago

The other thing I added when it started working was 'htaccess.RewriteBase' => '/', but I am not sure if that will work.

Was worth a try, but yea, it didn't do anything.

ne20002 commented 6 months ago

In my 28.0.5 behind a Nginx reverse proxy the checks for /.well-known/carddav and /.well-known/carddav work now after adding the trailing / on the redirect. It still complains about /.well-known/nodeinfo and /.well-known/webfinger where I don't have any redirect and forward as is to the Nextcloud server which is also a Nginx / fpm with the configuration as from the latest manual.

sn0n commented 6 months ago

In my 28.0.5

This is a 29 issue.

RealKoenisch commented 6 months ago

It works with the default setting. I had the problem before and send a solution by changing the number in the redirect statement. The final solution was disabeling all rewrite rules in .htaccess and copy the the content (only the few lines with the rewrites) from git. Works for Apache.

  RewriteEngine On
  RewriteCond %{HTTP_USER_AGENT} DavClnt
  RewriteRule ^$ /remote.php/webdav/ [L,R=302]
  RewriteRule .* - [env=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
  RewriteRule ^\.well-known/carddav /remote.php/dav/ [R=301,L]
  RewriteRule ^\.well-known/caldav /remote.php/dav/ [R=301,L]
  RewriteRule ^remote/(.*) remote.php [QSA,L]
  RewriteRule ^(?:build|tests|config|lib|3rdparty|templates)/.* - [R=404,L]
  RewriteRule ^\.well-known/(?!acme-challenge|pki-validation) /index.php [QSA,L]
  RewriteRule ^ocm-provider/?$ index.php [QSA,L]
  RewriteRule ^(?:\.(?!well-known)|autotest|occ|issue|indie|db_|console).* - [R=404,L]

KR

BernieO commented 6 months ago

The check also takes the trusted_domains into account, which should be declared without a subfolder and make the check pass.

@kesselb: this doesn't seem to work in my case. With the default configuration for nextcloud in a subdir, the /.well-known-check fails already on .well-known/webfinger

Nextcloud: 29.0.0 (located in a subdir of webroot) Webserver: nginx (default config for subdir from docs.nextcloud.com)

This is a fresh installation only for testing this, with no extra configuration, hence config.php is quite small:

$CONFIG = array (
  'passwordsalt' => '****',
  'secret' => '****',
  'trusted_domains' => 
  array (
    0 => 'subdir.webroot.de',
  ),
  'datadirectory' => '/path/to/subdir.webroot.de/nextcloud/data',
  'dbtype' => 'sqlite3',
  'version' => '29.0.0.19',
  'overwrite.cli.url' => 'https://subdir.webroot.de/nextcloud',
  'installed' => true,
  'instanceid' => '****',
);

The redirect is configured properly and works as it should:

root@server:/# curl -IL https://subdir.webroot.de/.well-known/webfinger

HTTP/2 301 
server: nginx
date: Sun, 05 May 2024 06:35:27 GMT
content-type: text/html
content-length: 162
location: https://subdir.webroot.de/nextcloud/index.php/.well-known/webfinger
referrer-policy: no-referrer
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-permitted-cross-domain-policies: none
x-xss-protection: 1; mode=block
strict-transport-security: max-age=15768000; includeSubDomains; preload
x-robots-tag: noindex, nofollow

HTTP/2 404 
server: nginx
date: Sun, 05 May 2024 06:35:27 GMT
content-type: application/json; charset=utf-8
content-length: 37
set-cookie: oc_sessionPassphrase=jno%2FSH6iYwVKsyRwVYw9UQlGEh8CYvu6VW9sDAnVQNJHSRAcaqEM86G%2BDW2cWjZZNqSxCRG5YWeFmevBNOJWvt1lGlqqvq4zLe7EQHaWblTjoOwIBq9JfnUTWmkQjKqs; path=/nextcloud; secure; HttpOnly; SameSite=Lax
set-cookie: nc_sameSiteCookielax=true; path=/nextcloud; httponly;secure; expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=lax
set-cookie: nc_sameSiteCookiestrict=true; path=/nextcloud; httponly;secure; expires=Fri, 31-Dec-2100 23:59:59 GMT; SameSite=strict
set-cookie: ocpe2vl716a4=qgon0iqhe10k2pfq31thdqq3r6; path=/nextcloud; secure; HttpOnly; SameSite=Lax
expires: Thu, 19 Nov 1981 08:52:00 GMT
pragma: no-cache
x-request-id: MceyZE07GX8qfHsxfVIH
cache-control: no-cache, no-store, must-revalidate
content-security-policy: default-src 'none';base-uri 'none';manifest-src 'self';frame-ancestors 'none'
feature-policy: autoplay 'none';camera 'none';fullscreen 'none';geolocation 'none';microphone 'none';payment 'none'
x-robots-tag: noindex, nofollow
x-nextcloud-well-known: 1
referrer-policy: no-referrer
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-permitted-cross-domain-policies: none
x-robots-tag: noindex, nofollow
x-xss-protection: 1; mode=block
root@server:/# curl -L https://subdir.webroot.de/.well-known/webfinger
{"message":"webfinger not supported"}

But the redirect from the webroot is not being tested as you can see in the nginx access log when visiting /nextcloud/settings/admin/overview with a webbrowser (filtered for user-agent-string Nextcloud Server Crawler):

1.2.3.4 - - [05/May/2024:08:21:18 +0200] "PROPFIND /nextcloud/remote.php/webdav HTTP/1.1" 401 426 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [05/May/2024:08:21:19 +0200] "HEAD /nextcloud/data/.ocdata HTTP/1.1" 404 0 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [05/May/2024:08:21:19 +0200] "HEAD /nextcloud/data/.ocdata HTTP/1.1" 404 0 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [05/May/2024:08:21:19 +0200] "HEAD /nextcloud/nextcloud/data/.ocdata HTTP/1.1" 404 0 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [05/May/2024:08:21:20 +0200] "HEAD /nextcloud/ocm-provider/ HTTP/1.1" 200 0 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [05/May/2024:08:21:20 +0200] "HEAD /nextcloud/ocs-provider/ HTTP/1.1" 200 0 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [05/May/2024:08:21:21 +0200] "GET /nextcloud/ HTTP/1.1" 302 5 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [05/May/2024:08:21:21 +0200] "GET /nextcloud/login HTTP/1.1" 200 5818 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [05/May/2024:08:21:21 +0200] "GET /nextcloud/.well-known/webfinger HTTP/1.1" 404 146 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [05/May/2024:08:21:21 +0200] "GET /nextcloud/.well-known/webfinger HTTP/1.1" 404 146 "-" "Nextcloud Server Crawler"

kesselb commented 6 months ago

/nextcloud/.well-known/webfinger

What happens if you curl this uri? The curl output from above shows a 301 to /nextcloud/index.php/.well-known/webfinger.

BernieO commented 6 months ago

/nextcloud/.well-known/webfinger What happens if you curl this uri?

root@server:~# curl -i https://subdir.webroot.de/nextcloud/.well-known/webfinger
HTTP/2 404 
server: nginx
date: Sun, 05 May 2024 09:48:31 GMT
content-type: text/html
content-length: 146
referrer-policy: no-referrer
x-content-type-options: nosniff
x-frame-options: SAMEORIGIN
x-permitted-cross-domain-policies: none
x-robots-tag: noindex, nofollow
x-xss-protection: 1; mode=block

<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
kesselb commented 6 months ago

That's the nginx 404 page which misses the x-nextcloud-well-known: 1 header and therefore the check fails. I guess you have to find out why this request is not forwarded to Nextcloud but handled directly by nginx.

kesselb commented 6 months ago

@BernieO I'm a bit confused by your configuration. Aren't pretty urls enabled by default when using nginx?

https://github.com/nextcloud/server/blob/10af08736e9094dd3f7240ec517cea9a0061d5b4/apps/settings/lib/SetupChecks/CheckServerResponseTrait.php#L69

I think the reason for the request without index.php is that we don't factor in setups without pretty urls here.

Edit: The request for /nextcloud/.well-known/webfinger is supposed to fail and therefore the 404 from nginx okay. I guess the request is initiated from overwrite.cli.url (the last case in test urls). The more important question is why the previous request for the trusted domains does not succeed. Is there no request for subdir.webroot.de/.well-known/webfinger in your logs?

BernieO commented 6 months ago

Is there no request for subdir.webroot.de/.well-known/webfinger in your logs?

No, there is not. I just double checked. Only requests for uris within subdir.webroot.de/nextcloud/ are in the nginx log with user-agent-string Nextcloud Server Crawler.

EDIT (06.05.2024):

I guess the request is initiated from overwrite.cli.url (the last case in test urls). The more important question is why the previous request for the trusted domains does not succeed.

I changed the overwrite.cli.url so that it is different from trusted_domains and added $host to the nginx access log to see the hostname of the request. The relevant lines of config.php are now:

  'trusted_domains' => 
  array (
    0 => 'subdir.webroot.de',
  ),
  'overwrite.cli.url' => 'https://overwritecliurl.webroot.de/nextcloud',

Your guess was correct: the second request for /.well-known/webfinger is initiated from overwrite.cli.url:

1.2.3.4 - - [06/May/2024:07:35:08 +0200] "subdir.webroot.de" "PROPFIND /nextcloud/remote.php/webdav HTTP/1.1" 401 426 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:07:35:10 +0200] "subdir.webroot.de" "HEAD /nextcloud/data/.ocdata HTTP/1.1" 404 0 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:07:35:10 +0200] "subdir.webroot.de" "HEAD /nextcloud/data/.ocdata HTTP/1.1" 404 0 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:07:35:10 +0200] "overwritecliurl.webroot.de" "HEAD /nextcloud/nextcloud/data/.ocdata HTTP/1.1" 400 0 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:07:35:11 +0200] "subdir.webroot.de" "HEAD /nextcloud/ocm-provider/ HTTP/1.1" 200 0 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:07:35:12 +0200] "subdir.webroot.de" "HEAD /nextcloud/ocs-provider/ HTTP/1.1" 200 0 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:07:35:12 +0200] "subdir.webroot.de" "GET /nextcloud/ HTTP/1.1" 302 5 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:07:35:12 +0200] "subdir.webroot.de" "GET /nextcloud/login HTTP/1.1" 200 5821 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:07:35:13 +0200] "subdir.webroot.de" "GET /nextcloud/.well-known/webfinger HTTP/1.1" 404 146 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:07:35:13 +0200] "overwritecliurl.webroot.de" "GET /nextcloud/.well-known/webfinger HTTP/1.1" 404 146 "-" "Nextcloud Server Crawler"

Is there anything else I can do to help debugging?

kesselb commented 6 months ago

I noticed two things:

1) We are using the trusted_domains without any "preprocessing". For example my dev-setup is tls-only. But the check request for the trusted domains goes to http:// because not protocol is given. overwriteprotocol is https and therefore the check for the trusted domains should also use tls.

2) The overwrite.cli.url includes the complete url including a webroot. We want to check if the redirect works and therefore need to strip the webroot. The patch below should fix that.

Index: apps/settings/lib/SetupChecks/CheckServerResponseTrait.php
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/apps/settings/lib/SetupChecks/CheckServerResponseTrait.php b/apps/settings/lib/SetupChecks/CheckServerResponseTrait.php
--- a/apps/settings/lib/SetupChecks/CheckServerResponseTrait.php    (revision fe4c1b28c7353deb7c7d3429a7490a5922bf1621)
+++ b/apps/settings/lib/SetupChecks/CheckServerResponseTrait.php    (date 1714992736792)
@@ -61,7 +61,7 @@
        $hosts = $this->config->getSystemValue('trusted_domains', []);
        $cliUrl = $this->config->getSystemValue('overwrite.cli.url', '');
        if ($cliUrl !== '') {
-           $hosts[] = $cliUrl;
+           $hosts[] = rtrim(str_replace($this->urlGenerator->getWebroot(), '', $cliUrl), '/');
        }

        $testUrls = array_merge(
BernieO commented 6 months ago

That worked for my testing instance in a subdirectory. The warning in the admin panel disappeared. Here is the nginx access log when visiting the admin settings overview:

1.2.3.4 - - [06/May/2024:15:26:35 +0200] "subdir.webroot.de" "PROPFIND /nextcloud/remote.php/webdav HTTP/1.1" 401 426 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:15:26:36 +0200] "subdir.webroot.de" "HEAD /nextcloud/data/.ocdata HTTP/1.1" 404 0 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:15:26:36 +0200] "subdir.webroot.de" "HEAD /nextcloud/data/.ocdata HTTP/1.1" 404 0 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:15:26:36 +0200] "subdir.webroot.de" "HEAD /nextcloud/data/.ocdata HTTP/1.1" 404 0 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:15:26:36 +0200] "overwritecliurl.webroot.de" "HEAD /nextcloud/data/.ocdata HTTP/1.1" 404 0 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:15:26:37 +0200] "subdir.webroot.de" "HEAD /nextcloud/ocm-provider/ HTTP/1.1" 200 0 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:15:26:37 +0200] "subdir.webroot.de" "HEAD /nextcloud/ocs-provider/ HTTP/1.1" 200 0 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:15:26:38 +0200] "subdir.webroot.de" "GET /nextcloud/ HTTP/1.1" 302 5 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:15:26:38 +0200] "subdir.webroot.de" "GET /nextcloud/login HTTP/1.1" 200 5816 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:15:26:38 +0200] "subdir.webroot.de" "GET /nextcloud/.well-known/webfinger HTTP/1.1" 404 146 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:15:26:38 +0200] "overwritecliurl.webroot.de" "GET /.well-known/webfinger HTTP/1.1" 301 162 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:15:26:38 +0200] "overwritecliurl.webroot.de" "GET /nextcloud/index.php/.well-known/webfinger HTTP/1.1" 404 37 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:15:26:38 +0200] "subdir.webroot.de" "GET /nextcloud/.well-known/nodeinfo HTTP/1.1" 404 146 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:15:26:38 +0200] "overwritecliurl.webroot.de" "GET /.well-known/nodeinfo HTTP/1.1" 301 162 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:15:26:38 +0200] "overwritecliurl.webroot.de" "GET /nextcloud/index.php/.well-known/nodeinfo HTTP/1.1" 404 36 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:15:26:38 +0200] "subdir.webroot.de" "PROPFIND /nextcloud/.well-known/caldav HTTP/1.1" 404 146 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:15:26:38 +0200] "overwritecliurl.webroot.de" "PROPFIND /.well-known/caldav HTTP/1.1" 301 162 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:15:26:39 +0200] "overwritecliurl.webroot.de" "GET /nextcloud/remote.php/dav/ HTTP/1.1" 401 569 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:15:26:40 +0200] "subdir.webroot.de" "PROPFIND /nextcloud/.well-known/carddav HTTP/1.1" 404 146 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:15:26:40 +0200] "overwritecliurl.webroot.de" "PROPFIND /.well-known/carddav HTTP/1.1" 301 162 "-" "Nextcloud Server Crawler"
1.2.3.4 - - [06/May/2024:15:26:40 +0200] "overwritecliurl.webroot.de" "GET /nextcloud/remote.php/dav/ HTTP/1.1" 401 569 "-" "Nextcloud Server Crawler"

It also works in a production instance (also located in a subdirectory).Thanks a lot!

BernieO commented 6 months ago

@kesselb how do we proceed? Should I open a pull request for the code change which seems to solve the problem for subdir installations??

kesselb commented 6 months ago

Pull Request to update the documentation for Caddy: https://github.com/nextcloud/documentation/pull/11799

Should I open a pull request for the code change which seems to solve the problem for subdir installations??

The code is used by multiple setup checks, and thus I don't know if it's good to strip the webroot for every case. Do you see any new failing setup checks? ;)

BernieO commented 6 months ago

The code is used by multiple setup checks, and thus I don't know if it's good to strip the webroot for every case.

I see.

Do you see any new failing setup checks?

No, not regarding .well-known uris with your suggested patch from https://github.com/nextcloud/server/issues/45033#issuecomment-2095742325

Lawkss commented 6 months ago

With the suggested patch I fail on caldav, with the defaults I fail on webfinger.

I have had this in the conf for apache for many past versions of nextcloud and it has always worked fine. with cloud being my subdirectory. This is new to 29.

Redirect 301 /.well-known/carddav /cloud/remote.php/dav Redirect 301 /.well-known/caldav /cloud/remote.php/dav Redirect 301 /.well-known/webfinger /cloud/index.php/.well-known/webfinger Redirect 301 /.well-known/nodeinfo /cloud/index.php/.well-known/nodeinfo

MichaIng commented 6 months ago

I am faced with this issue as well + we have reports from our users. In my case, it is Apache2. In all our cases, Nextcloud is installed in a /nextcloud sub directory of the webroot, hence redirects are setup for all webserver with a separate config snipped:

Redirect 301 /.well-known/carddav /nextcloud/remote.php/dav
Redirect 301 /.well-known/caldav  /nextcloud/remote.php/dav

Checking the .htaccess, the first thing I tried was adding the trailing slash, which is already in .htaccess since a longer time, but was never required before. And also now, for me, it does not solve the issue. A check verifies that the URL is successfully redirected:

root@micha:~# curl -I https://my.domain.org/.well-known/caldav
HTTP/2 301
x-content-type-options: nosniff
x-frame-options: sameorigin
x-robots-tag: noindex, nofollow
x-permitted-cross-domain-policies: none
referrer-policy: no-referrer
content-security-policy: upgrade-insecure-requests; default-src 'none'; frame-ancestors 'none'; base-uri 'none'; form-action 'none'
permissions-policy: accelerometer=(), autoplay=(), browsing-topics=(), camera=(), encrypted-media=(), fullscreen=(), geolocation=(), gyroscope=(), magnetometer=(), microphone=(), midi=(), payment=(), usb=(), screen-wake-lock=()
strict-transport-security: max-age=15552000; includeSubDomains; preload
location: https://my.domain.org/nextcloud/remote.php/dav/
content-type: text/html; charset=iso-8859-1
date: Sun, 26 May 2024 09:10:59 GMT
server: Apache

Since something obviously has changed regarding this check, aside of the expected trailing slash, probably it does not respect NC in sub directories anymore? I'll have a look into the code when I find time.

Also, at least the Nginx config in official docs requires an update then.

kesselb commented 6 months ago

@MichaIng https://github.com/nextcloud/server/issues/45033#issuecomment-2095742325

MichaIng commented 6 months ago

Ah right, sorry for not reading everything carefully enough. The patch works, the trailing slash is still required. Does Nextcloud handle remote.php/dav and remote.php/dav/ differently, or could the trailing slash be removed from (or added to) all resulting URLs, to avoid the need to update all webserver configs (including the docs)? EDIT: I see, docs changes are already done, Nginx had the slash already.

memesalot commented 6 months ago

This works for me in NginxProxyManager:

location /.well-known/carddav {
    return 301 $scheme://$host/remote.php/dav/;
}

location /.well-known/caldav {
    return 301 $scheme://$host/remote.php/dav/;
}
197788 commented 6 months ago

I have the same setup as the author that opened this issue (nginx, postgres) and no matter how I try to tweak the nginx settings with slashes or without, it gives the famous warning:

[webdav] Debug: No public access to this resource., No 'Authorization: Basic' header found. Either the client didn't send one, or the server is misconfigured, No 'Authorization: Bearer' header found. Either the client didn't send one, or the server is mis-configured, No 'Authorization: Basic' header found. Either the client didn't send one, or the server is misconfigured GET /remote.php/dav/ from 127.0.0.1 by -- at May 31, 2024, 11:55:43 PM

MichaIng commented 6 months ago

@197788 Your issue and error message are however entirely different, not even related to CalDAV or CardDAV but webdav. Please verify first with another freshly configured webdav client, whether you can replicate it, because it looks like a client issue. And in case, open a new issue.

itsthejb commented 6 months ago

In my case, the issue was only the missing trailing slash as https://github.com/nextcloud/server/issues/45033#issuecomment-2079306503

cebix commented 5 months ago

I ran into the same problem (warning about /.well-known/caldav on the config page) with Apache, and fixed it by manually creating a directory named .well-known inside the Nextcloud installation directory. Without this, Apache would give an error AH01630: client denied by server configuration when trying to access the URL, and the rewrites in the .htaccess file would not trigger. Nextcloud is installed in the webroot, and my Apache config does not contain any explicit Redirect 301 directives as mentioned above.