search

nextcloud / server

☁️ Nextcloud server, a safe home for all your data
https://nextcloud.com
GNU Affero General Public License v3.0
27.38k stars 4.07k forks source link

[Bug]: isHtaccessWorking check fails on IDN domains #45724

Closed Sprinterfreak closed 5 months ago

Sprinterfreak commented 5 months ago

⚠️ This issue respects the following points: ⚠️

Bug description

On IDN instances, the check falsely reports

Your data directory and files are probably accessible from the internet because the .htaccess file does not work.

My guess is, that https://github.com/nextcloud/server/blob/49c42c36aedf69ea8393c6cbc4387e04adc84761/lib/private/legacy/OC_Util.php#L883 fails to tetch the url if the domain contains umlauts, which corrupts the test condition.

Steps to reproduce

  1. Get yourself an IDN domain and host a Nextcloud instance on it.
  2. Navigate to Verwaltungseinstellungen -> Übersicht

Expected behavior

The check should not be displayed if there is no issue

Installation method

Community Manual installation with Archive

Nextcloud Server version

29

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.2

Web server

Nginx

Database engine version

MySQL

Is this bug present after an update or on a fresh install?

Upgraded to a MAJOR version (ex. 22 to 23)

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

Configuration report

{
    "system": {
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "localhost",
            "cloud.xn--tst-qla.com"
        ],
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "overwritehost": "cloud.xn--tst-qla.com",
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "dbtype": "pgsql",
        "version": "29.0.2.2",
        "overwrite.cli.url": "https:\/\/cloud.xn--tst-qla.com",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "memcache.local": "\\OC\\Memcache\\Redis",
        "memcache.distributed": "\\OC\\Memcache\\Redis",
        "memcache.locking": "\\OC\\Memcache\\Redis",
        "redis": {
            "host": "***REMOVED SENSITIVE VALUE***",
            "port": "0"
        },
        "filelocking.enabled": "true",
        "preview_format": "webp",
        "enable_previews": "true",
        "enabledPreviewProviders": [
            "OC\\Preview\\PNG",
            "OC\\Preview\\JPEG",
            "OC\\Preview\\GIF",
            "OC\\Preview\\BMP",
            "OC\\Preview\\XBitmap",
            "OC\\Preview\\Movie",
            "OC\\Preview\\PDF",
            "OC\\Preview\\MP3",
            "OC\\Preview\\TXT",
            "OC\\Preview\\MarkDown",
            "OC\\Preview\\TIFF"
        ],
        "preview_max_scale_factor": "1",
        "preview_max_memory": 256,
        "auth.bruteforce.protection.enabled": "true",
        "trashbin_retention_obligation": "auto,7",
        "skeletondirectory": "",
        "defaultapp": "file",
        "activity_expire_days": "14",
        "updater.release.channel": "stable",
        "mail_smtpmode": "smtp",
        "mail_smtpsecure": "tls",
        "mail_sendmailmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpport": "25",
        "maintenance": false,
        "theme": "",
        "loglevel": 0,
        "default_phone_region": "de",
        "check_for_working_wellknown_setup": false,
        "check_for_working_htaccess": true,
        "allow_local_remote_servers": true,
        "memories.exiftool": "\/var\/www\/nextcloud\/apps\/memories\/bin-ext\/exiftool-amd64-glibc",
        "memories.exiftool.tmp": "\/var\/nc-data\/__memories",
        "memories.vod.path": "\/var\/www\/nextcloud\/apps\/memories\/bin-ext\/go-vod-amd64",
        "memories.vod.ffmpeg": "\/usr\/bin\/ffmpeg",
        "memories.vod.ffprobe": "\/usr\/bin\/ffprobe",
        "memories.vod.tempdir": "\/var\/nc-data\/__memories_vodtemp",
        "facerecognition.external_model_url": "http:\/\/192.168.2.88:5000",
        "facerecognition.external_model_api_key": "ODeQIC9xsa0WK901UrgCJUybCBeV4vZbOonDgBsjtBA=",
        "maintenance_window_start": 4,
        "memories.db.triggers.fcu": true
    },
    "apps": {
        "activity": {
            "types": "filesystem",
            "enabled": "yes",
            "installed_version": "2.21.1",
            "notify_notification_file_changed": "1",
            "notify_notification_group_settings": "0",
            "notify_email_group_settings": "1",
            "notify_notification_security": "1",
            "notify_notification_personal_settings": "0",
            "notify_notification_favorite": "0",
            "notify_email_file_changed": "1",
            "notify_notification_file_favorite_changed": "0",
            "notify_notification_comments": "0",
            "notify_email_comments": "0",
            "notify_notification_calendar": "0",
            "notify_email_calendar": "0",
            "notify_notification_calendar_event": "0",
            "notify_notification_shared": "1",
            "notify_email_calendar_event": "0",
            "notify_notification_calendar_todo": "0",
            "notify_email_calendar_todo": "0",
            "notify_notification_contacts": "0",
            "notify_email_contacts": "0",
            "notify_notification_remote_share": "0",
            "notify_email_remote_share": "0",
            "notify_notification_public_links": "0",
            "notify_email_public_links": "0",
            "notify_notification_phonetrack": "0",
            "notify_email_phonetrack": "0",
            "notify_notification_phonetrack_geofence_event": "0",
            "notify_email_phonetrack_geofence_event": "0",
            "notify_notification_phonetrack_proximity_event": "0",
            "notify_email_phonetrack_proximity_event": "0",
            "notify_notification_systemtags": "0",
            "notify_email_systemtags": "0",
            "notify_setting_batchtime": "3600",
            "notify_setting_self": "0",
            "notify_setting_selfemail": "0",
            "notify_email_shared": "1"
        },
        "admin_audit": {
            "types": "logging",
            "enabled": "yes",
            "installed_version": "1.19.0"
        },
        "backgroundjob": {
            "lastjob": "107282"
        },
        "calendar": {
            "types": "",
            "enabled": "yes",
            "installed_version": "4.7.6"
        },
        "camerarawpreviews": {
            "types": "filesystem",
            "enabled": "yes",
            "installed_version": "0.8.5"
        },
        "checksum": {
            "installed_version": "1.2.3",
            "types": "filesystem",
            "enabled": "no"
        },
        "circles": {
            "migration_22": "1",
            "migration_22_1": "1",
            "loopback_tmp_scheme": "https",
            "types": "filesystem,dav",
            "enabled": "yes",
            "migration_run": "0",
            "installed_version": "29.0.0-dev",
            "maintenance_run": "0",
            "maintenance_update": "{\"maximum\":3,\"3\":1717848614,\"2\":1717849655,\"1\":1717849663,\"4\":1655867341,\"5\":1655525701}"
        },
        "cloud_federation_api": {
            "types": "filesystem",
            "enabled": "yes",
            "installed_version": "1.12.0"
        },
        "collectives": {
            "types": "filesystem",
            "installed_version": "2.9.2",
            "enabled": "no"
        },
        "comments": {
            "types": "logging",
            "enabled": "yes",
            "installed_version": "1.19.0"
        },
        "contacts": {
            "enabled": "yes",
            "types": "dav",
            "installed_version": "6.0.0"
        },
        "contactsinteraction": {
            "types": "dav",
            "enabled": "yes",
            "installed_version": "1.10.0"
        },
        "core": {
            "installedat": "1616774785.4071",
            "vendor": "nextcloud",
            "public_files": "files_sharing\/public.php",
            "emailTestSuccessful": "1",
            "public_webdav": "dav\/appinfo\/v1\/publicwebdav.php",
            "backgroundjobs_mode": "cron",
            "theming.variables": "8a4dfc62e6a15302e611f671613c3492",
            "moveavatarsdone": "yes",
            "previewsCleanedUp": "1",
            "files_metadata_installed": "1",
            "lastupdatedat": "1717847128",
            "lastcron": "1717849709",
            "lastupdateResult": "[]",
            "updater.secret.created": "1717760525",
            "oc.integritycheck.checker": "[]",
            "files_metadata": "{\"photos-original_date_time\":{\"value\":null,\"type\":\"int\",\"etag\":\"\",\"indexed\":true,\"editPermission\":0},\"photos-size\":{\"value\":null,\"type\":\"array\",\"etag\":\"\",\"indexed\":false,\"editPermission\":0},\"photos-exif\":{\"value\":null,\"type\":\"array\",\"etag\":\"\",\"indexed\":false,\"editPermission\":0},\"photos-ifd0\":{\"value\":null,\"type\":\"array\",\"etag\":\"\",\"indexed\":false,\"editPermission\":0},\"photos-gps\":{\"value\":null,\"type\":\"array\",\"etag\":\"\",\"indexed\":false,\"editPermission\":0},\"photos-place\":{\"value\":null,\"type\":\"string\",\"etag\":\"\",\"indexed\":true,\"editPermission\":0},\"blurhash\":{\"value\":null,\"type\":\"string\",\"etag\":\"ebd5f32ebeb8b664ae5a66a6dd84dae0\",\"indexed\":false,\"editPermission\":0}}"
        },
        "dashboard": {
            "installed_version": "7.1.0",
            "types": "",
            "enabled": "no"
        },
        "dav": {
            "types": "filesystem",
            "enabled": "yes",
            "regeneratedBirthdayCalendarsForYearFix": "yes",
            "chunks_migrated": "1",
            "buildCalendarSearchIndex": "yes",
            "buildCalendarReminderIndex": "yes",
            "installed_version": "1.30.1"
        },
        "drawio": {
            "types": "filesystem",
            "installed_version": "3.0.2",
            "enabled": "no"
        },
        "facerecognition": {
            "model": "5",
            "installed_version": "0.9.31",
            "min_image_size": "128",
            "handle_external_files": "true",
            "min_confidence": "0.8",
            "assigned_memory": "1073741824",
            "types": "filesystem",
            "enabled": "no",
            "analysis_image_area": "7713600",
            "max_image_area": "786432",
            "handle_shared_files": "true"
        },
        "federatedfilesharing": {
            "types": "",
            "enabled": "yes",
            "installed_version": "1.19.0"
        },
        "federation": {
            "types": "authentication",
            "enabled": "yes",
            "installed_version": "1.19.0"
        },
        "files": {
            "types": "filesystem",
            "enabled": "yes",
            "installed_version": "2.1.0"
        },
        "files_downloadlimit": {
            "installed_version": "2.0.0",
            "types": "",
            "enabled": "yes"
        },
        "files_markdown": {
            "enabled": "no",
            "types": "",
            "installed_version": "2.4.1"
        },
        "files_pdfviewer": {
            "types": "",
            "enabled": "yes",
            "installed_version": "2.10.0"
        },
        "files_reminders": {
            "installed_version": "1.1.0",
            "types": "",
            "enabled": "no"
        },
        "files_rightclick": {
            "types": "",
            "installed_version": "1.6.0",
            "enabled": "no"
        },
        "files_sharing": {
            "types": "filesystem",
            "enabled": "yes",
            "installed_version": "1.21.0",
            "incoming_server2server_group_share_enabled": "yes",
            "outgoing_server2server_group_share_enabled": "yes"
        },
        "files_trashbin": {
            "types": "filesystem,dav",
            "enabled": "yes",
            "installed_version": "1.19.0"
        },
        "files_versions": {
            "types": "filesystem,dav",
            "enabled": "yes",
            "installed_version": "1.22.0"
        },
        "firstrunwizard": {
            "installed_version": "2.10.0",
            "types": "logging",
            "enabled": "no"
        },
        "groupfolders": {
            "types": "filesystem,dav",
            "enabled": "yes",
            "installed_version": "17.0.0"
        },
        "keeweb": {
            "enabled": "no",
            "types": "",
            "installed_version": "0.6.13"
        },
        "logreader": {
            "enabled": "yes",
            "types": "logging",
            "installed_version": "2.14.0",
            "live": "1",
            "shownLevels": "[2,3,4]"
        },
        "lookup_server_connector": {
            "types": "authentication",
            "enabled": "yes",
            "installed_version": "1.17.0"
        },
        "memories": {
            "types": "",
            "enabled": "yes",
            "last_index_job_status": "Indexing completed successfully",
            "last_index_job_status_type": "success",
            "last_index_job_duration": "13.37",
            "installed_version": "7.3.1",
            "last_index_job_start": "1717849328"
        },
        "nextcloud_announcements": {
            "types": "logging",
            "installed_version": "1.17.0",
            "enabled": "no",
            "pub_date": "Thu, 24 Oct 2019 00:00:00 +0200"
        },
        "notifications": {
            "types": "logging",
            "enabled": "yes",
            "sound_notification": "no",
            "sound_talk": "no",
            "setting_batchtime": "4",
            "installed_version": "2.17.0"
        },
        "oauth2": {
            "types": "authentication",
            "enabled": "yes",
            "installed_version": "1.17.0"
        },
        "password_policy": {
            "types": "authentication",
            "enabled": "yes",
            "installed_version": "1.19.0",
            "minLength": "6",
            "enforceSpecialCharacters": "1",
            "enforceNumericCharacters": "0",
            "enforceHaveIBeenPwned": "1"
        },
        "phonetrack": {
            "enabled": "yes",
            "types": "",
            "installed_version": "0.8.1"
        },
        "photos": {
            "enabled": "yes",
            "types": "dav,authentication",
            "installed_version": "2.5.0",
            "lastPlaceMappedUser": "jan",
            "lastPlaceMappingDone": "true"
        },
        "preview": {
            "format": "webp",
            "preview_format": "webp"
        },
        "previewgenerator": {
            "enabled": "yes",
            "types": "filesystem",
            "installed_version": "5.5.0"
        },
        "privacy": {
            "types": "",
            "enabled": "yes",
            "installed_version": "1.13.0",
            "readableLocation": "de"
        },
        "provisioning_api": {
            "types": "prevent_group_restriction",
            "enabled": "yes",
            "installed_version": "1.19.0"
        },
        "ransomware_protection": {
            "installed_version": "1.14.0",
            "types": "logging",
            "enabled": "no"
        },
        "recognize": {
            "clusterFaces.status": "true",
            "faces.batchSize": "100",
            "nice_binary": "\/usr\/bin\/nice",
            "node_binary": "\/var\/www\/nextcloud\/apps\/recognize\/bin\/node",
            "landmarks.enabled": "false",
            "musicnn.enabled": "false",
            "movinet.enabled": "false",
            "landmarks.batchSize": "20",
            "movinet.batchSize": "5",
            "musicnn.batchSize": "20",
            "nice_value": "0",
            "imagenet.enabled": "true",
            "imagenet.batchSize": "10",
            "tensorflow.gpu": "false",
            "tensorflow.purejs": "true",
            "tensorflow.cores": "4",
            "faces.status": "true",
            "types": "filesystem,dav",
            "concurrency.enabled": "true",
            "imagenet.status": "true",
            "enabled": "yes",
            "faces.enabled": "true",
            "clusterFaces.lastRun": "1717505526",
            "imagenet.lastFile": "1717787480",
            "installed_version": "7.0.0",
            "faces.lastFile": "1717505350"
        },
        "recommendations": {
            "types": "",
            "enabled": "yes",
            "installed_version": "2.1.0"
        },
        "related_resources": {
            "types": "",
            "enabled": "yes",
            "installed_version": "1.4.0"
        },
        "richdocuments": {
            "canonical_webroot": "",
            "wopi_allowlist": "127.0.0.1",
            "wopi_url": "https:\/\/cloud.xn--tst-qla.com",
            "disable_certificate_verification": "yes",
            "public_wopi_url": "https:\/\/cloud.xn--tst-qla.com",
            "types": "prevent_group_restriction",
            "installed_version": "8.4.2",
            "enabled": "yes"
        },
        "serverinfo": {
            "types": "",
            "enabled": "yes",
            "installed_version": "1.19.0",
            "cached_count_storages": "19",
            "cached_count_filecache": "173816"
        },
        "settings": {
            "types": "",
            "enabled": "yes",
            "installed_version": "1.12.0"
        },
        "sharebymail": {
            "types": "filesystem",
            "enabled": "yes",
            "installed_version": "1.19.0"
        },
        "shifts": {
            "installed_version": "1.8.6",
            "types": "",
            "calendarName": "ShiftsCalendar",
            "adminGroup": "ShiftsAdmin",
            "enabled": "no",
            "organizerName": "jan",
            "skillGroups": "[{\"id\":\"0\",\"name\":\"Level 1\"},{\"id\":\"1\",\"name\":\"Bla\"}]",
            "organizerEmail": "jan@saenet.de",
            "shiftWorkerGroup": "ShiftsAdmin"
        },
        "support": {
            "types": "session",
            "enabled": "no",
            "SwitchUpdaterServerHasRun": "yes",
            "installed_version": "1.11.0"
        },
        "survey_client": {
            "installed_version": "1.9.0",
            "types": "",
            "enabled": "no"
        },
        "systemtags": {
            "types": "logging",
            "enabled": "yes",
            "installed_version": "1.19.0"
        },
        "text": {
            "types": "dav",
            "enabled": "yes",
            "installed_version": "3.10.0"
        },
        "theming": {
            "url": "***REMOVED SENSITIVE VALUE***",
            "types": "logging",
            "enabled": "yes",
            "installed_version": "2.4.0",
            "cachebuster": "5",
            "name": "S\u00e4mann's Cloud",
            "slogan": "***REMOVED SENSITIVE VALUE***"
        },
        "twofactor_backupcodes": {
            "types": "",
            "enabled": "yes",
            "installed_version": "1.18.0"
        },
        "updatenotification": {
            "types": "",
            "enabled": "yes",
            "checksum": "1.2.3",
            "related_resources": "1.0.4",
            "drawio": "2.1.4",
            "richdocuments": "8.3.7",
            "groupfolders": "17.0.1",
            "keeweb": "0.6.13",
            "contacts": "5.5.3",
            "phonetrack": "0.8.1",
            "update_check_errors": "0",
            "installed_version": "1.19.1",
            "collectives": "2.6.1",
            "calendar": "4.7.6",
            "memories": "7.3.1",
            "previewgenerator": "5.5.0",
            "camerarawpreviews": "0.8.5",
            "recognize": "6.1.1",
            "core": "29.0.2.2"
        },
        "user_status": {
            "types": "",
            "enabled": "yes",
            "installed_version": "1.9.0"
        },
        "viewer": {
            "types": "",
            "enabled": "yes",
            "installed_version": "2.3.0"
        },
        "weather_status": {
            "types": "",
            "installed_version": "1.8.0",
            "enabled": "no"
        },
        "workflowengine": {
            "types": "filesystem",
            "enabled": "yes",
            "installed_version": "2.11.0"
        }
    }
}

List of activated Apps

Enabled:
  - activity: 2.21.1
  - admin_audit: 1.19.0
  - calendar: 4.7.6
  - camerarawpreviews: 0.8.5
  - circles: 29.0.0-dev
  - cloud_federation_api: 1.12.0
  - comments: 1.19.0
  - contacts: 6.0.0
  - contactsinteraction: 1.10.0
  - dav: 1.30.1
  - federatedfilesharing: 1.19.0
  - federation: 1.19.0
  - files: 2.1.0
  - files_downloadlimit: 2.0.0
  - files_pdfviewer: 2.10.0
  - files_sharing: 1.21.0
  - files_trashbin: 1.19.0
  - files_versions: 1.22.0
  - groupfolders: 17.0.0
  - logreader: 2.14.0
  - lookup_server_connector: 1.17.0
  - memories: 7.3.1
  - notifications: 2.17.0
  - oauth2: 1.17.0
  - password_policy: 1.19.0
  - phonetrack: 0.8.1
  - photos: 2.5.0
  - previewgenerator: 5.5.0
  - privacy: 1.13.0
  - provisioning_api: 1.19.0
  - recognize: 7.0.0
  - recommendations: 2.1.0
  - related_resources: 1.4.0
  - richdocuments: 8.4.2
  - serverinfo: 1.19.0
  - settings: 1.12.0
  - sharebymail: 1.19.0
  - systemtags: 1.19.0
  - text: 3.10.0
  - theming: 2.4.0
  - twofactor_backupcodes: 1.18.0
  - updatenotification: 1.19.1
  - user_status: 1.9.0
  - viewer: 2.3.0
  - workflowengine: 2.11.0
Disabled:
  - bruteforcesettings: 2.9.0
  - dashboard: 7.9.0 (installed 7.1.0)
  - encryption: 2.17.0
  - files_external: 1.21.0
  - files_reminders: 1.2.0 (installed 1.1.0)
  - firstrunwizard: 2.18.0 (installed 2.10.0)
  - nextcloud_announcements: 1.18.0 (installed 1.17.0)
  - support: 1.12.0 (installed 1.11.0)
  - survey_client: 1.17.0 (installed 1.9.0)
  - suspicious_login: 7.0.0
  - twofactor_totp: 11.0.0-dev
  - user_ldap: 1.20.0
  - weather_status: 1.9.0 (installed 1.8.0)

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

No response

Additional info

No response

joshtrichards commented 5 months ago

The particular check that triggers that error is this one:

https://github.com/nextcloud/server/blob/49c42c36aedf69ea8393c6cbc4387e04adc84761/apps/settings/lib/SetupChecks/DataDirectoryProtected.php#L42-L61

It's a little different than the install time check (the one you found).

Also, in v29 all these checks are server-side now and they check every one of your configured trusted_domains and the overwrite.cli.url value.

In this case, that check only comes up positive if it's getting a 200 (OK) response from one of those URLs.

Can you check what accessing https://[each of your domains]/[datadirectory]/.ocdata via curl (or equivalent) from your Nextcloud server itself returns (container not host, if using containers)?

Sprinterfreak commented 5 months ago

So I tested every IP and every fqdn possible, http and https. http just redirects (301) to https like configured. https results either in 403 nginx error-page or the front-controller rendering a 404 Page not found. I mean, the data directory resides outside the webroot. I can't imagine how nginx should be able to get to it. There is no symlink to the data directory in the webroot either. Even tested with curl --head like the check does - no difference, can't get 200 anywhere. I'am not using containers.

Sprinterfreak commented 5 months ago

Ah wait. I hacked a debug print into the response check. The affected $dataURL reads "//var/nc-data/.ocdata" which gets read from filesystem directly. I guess thats not anticipated to happen?

kesselb commented 5 months ago

Same as https://github.com/nextcloud/server/issues/45087?

Sprinterfreak commented 5 months ago

@kesselb You are right. Closing this because of duplicate of #45087