search

nextcloud / server

☁️ Nextcloud server, a safe home for all your data
https://nextcloud.com
GNU Affero General Public License v3.0
27.52k stars 4.08k forks source link

[Bug]: Error PHP unserialize #47879

Open punkyard opened 2 months ago

punkyard commented 2 months ago

⚠️ This issue respects the following points: ⚠️

Bug description

Hi I would like to report a repeated error I can find in NC logs. I'm afraid I can't evaluate its consequences, many errors arrived at the same time: locked files, antivirus, SQL and redis server.

Steps to reproduce

find logs in NC Logging:

[PHP] Error: unserialize(): Error at offset 40 of 43 bytes at /var/www/html/apps/dav/lib/DAV/CustomPropertiesBackend.php#574
    PROPFIND /remote.php/dav/addressbooks/users/UserName/
    from 185.252.235.96 by UserName at 8 sept. 2024, 19:22:05

Expected behavior

no error

Nextcloud Server version

29

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.2

Web server

Nginx

Database engine version

PostgreSQL

Is this bug present after an update or on a fresh install?

Updated from a MINOR version (ex. 28.0.1 to 28.0.2)

Are you using the Nextcloud Server Encryption module?

Encryption is Disabled

What user-backends are you using?

Configuration report

No response

List of activated Apps

No response

Nextcloud Signing status

No response

Nextcloud Logs

{"reqId":"YizWhMVEux6WGmu7Clxp","level":3,"time":"2024-09-08T17:22:05+00:00","remoteAddr":"185.252.235.96","user":"UserName","app":"PHP","method":"PROPFIND","url":"/remote.php/dav/addressbooks/users/UserName/","message":"unserialize(): Error at offset 40 of 43 bytes at /var/www/html/apps/dav/lib/DAV/CustomPropertiesBackend.php#574","userAgent":"Mac OS X/10.15.7 (19H2026) AddressBookCore/1","version":"29.0.4.1","data":{"app":"PHP"},"id":"66e090d696463"}


### Additional info

#Server configuration detail
Operating system: Linux 5.10.0-32-amd64 [#](https://notre.rez0.net/s/ztLijxYSegjCmjt#h-server-configuration-detail)1 SMP Debian 5.10.223-1 (2024-08-10) x86_64

Webserver: Apache/2.4.62 (Unix) (fpm-fcgi)

Database: pgsql PostgreSQL 16.3 on x86_64-pc-linux-musl, compiled by gcc (Alpine 13.2.1_git20240309) 13.2.1 20240309, 64-bit

PHP version: 8.2.21

Modules loaded: Core, date, libxml, openssl, pcre, sqlite3, zlib, ctype, curl, dom, fileinfo, filter, hash, iconv, json, mbstring, SPL, session, PDO, pdo_sqlite, bz2, posix, random, readline, Reflection, standard, SimpleXML, tokenizer, xml, xmlreader, xmlwriter, mysqlnd, cgi-fcgi, apcu, bcmath, Phar, exif, ftp, gd, gmp, igbinary, imagick, imap, intl, ldap, memcached, pcntl, pdo_pgsql, pgsql, redis, smbclient, sodium, sysvsem, zip, libsmbclient, Zend OPcache

Nextcloud version: 29.0.4 - 29.0.4.1

Updated from an older Nextcloud/ownCloud or fresh install:

Where did you install Nextcloud from: unknown

<details><summary>Signing status</summary>

[]
</details>

<details><summary>List of activated apps</summary>

Enabled:
 - activity: 2.21.1
 - admin_audit: 1.19.0
 - announcementcenter: 6.8.1
 - auto_groups: 1.5.3
 - bruteforcesettings: 2.9.0
 - calendar: 4.7.16
 - cfg_share_links: 5.1.3
 - circles: 29.0.0-dev
 - cloud_federation_api: 1.12.0
 - collectives: 2.14.3
 - comments: 1.19.0
 - contacts: 6.0.0
 - contactsinteraction: 1.10.0
 - dashboard: 7.9.0
 - dav: 1.30.1
 - deck: 1.13.1
 - event_update_notification: 2.4.0
 - external: 5.4.0
 - federatedfilesharing: 1.19.0
 - files: 2.1.0
 - files_antivirus: 5.5.7
 - files_downloadlimit: 2.0.0
 - files_pdfviewer: 2.10.0
 - files_reminders: 1.2.0
 - files_sharing: 1.21.0
 - files_trashbin: 1.19.0
 - files_versions: 1.22.0
 - forms: 4.2.4
 - group_default_quota: 0.1.10
 - groupfolders: 17.0.3
 - integration_excalidraw: 2.2.0
 - integration_youtube: 0.3.0
 - logreader: 2.14.0
 - lookup_server_connector: 1.17.0
 - mail: 3.7.8
 - money: 0.28.0
 - nextcloud-aio: 0.6.0
 - nextcloud_announcements: 1.18.0
 - notes: 4.10.1
 - notifications: 2.17.0
 - notify_push: 0.7.0
 - oauth2: 1.17.0
 - password_policy: 1.19.0
 - polls: 7.2.2
 - provisioning_api: 1.19.0
 - quota_warning: 1.20.0
 - registration: 2.4.0
 - related_resources: 1.4.0
 - richdocuments: 8.4.6
 - settings: 1.12.0
 - sharebymail: 1.19.0
 - side_menu: 3.13.1
 - spreed: 19.0.8
 - support: 1.12.0
 - suspicious_login: 7.0.0
 - tasks: 0.16.1
 - terms_of_service: 2.5.0
 - text: 3.10.1
 - theming: 2.4.0
 - timemanager: 0.3.15
 - twofactor_backupcodes: 1.18.0
 - unroundedcorners: 1.1.3
 - user_status: 1.9.0
 - viewer: 2.3.0
 - welcome: 1.2.0
 - workflowengine: 2.11.0
Disabled:
 - encryption
 - federation: 1.17.0
 - files_external
 - firstrunwizard: 2.14.0
 - impersonate: 1.16.0
 - photos: 2.3.0
 - privacy: 1.13.0
 - recommendations: 1.4.0
 - serverinfo: 1.16.0
 - survey_client: 1.13.0
 - systemtags: 1.18.0
 - twofactor_totp: 7.0.0
 - user_ldap
 - weather_status: 1.5.0

</details>

<details><summary>Configuration (config/config.php)</summary>

{
    "memcache.local": "\\OC\\Memcache\\APCu",
    "apps_paths": [
        {
            "path": "\/var\/www\/html\/apps",
            "url": "\/apps",
            "writable": false
        },
        {
            "path": "\/var\/www\/html\/custom_apps",
            "url": "\/custom_apps",
            "writable": true
        }
    ],
    "memcache.distributed": "\\OC\\Memcache\\Redis",
    "memcache.locking": "\\OC\\Memcache\\Redis",
    "redis": {
        "host": "***REMOVED SENSITIVE VALUE***",
        "password": "***REMOVED SENSITIVE VALUE***",
        "port": 6379
    },
    "overwriteprotocol": "https",
    "passwordsalt": "***REMOVED SENSITIVE VALUE***",
    "secret": "***REMOVED SENSITIVE VALUE***",
    "trusted_domains": [
        "localhost",
        "notre.rez0.net",
        "kolab.koraland.net"
    ],
    "datadirectory": "***REMOVED SENSITIVE VALUE***",
    "skeletondirectory": "\/var\/lib\/docker\/volumes\/nextcloud_aio_nextcloud\/_data\/skeleton",
    "dbtype": "pgsql",
    "version": "29.0.4.1",
    "dbname": "***REMOVED SENSITIVE VALUE***",
    "dbhost": "***REMOVED SENSITIVE VALUE***",
    "dbport": "",
    "dbtableprefix": "oc_",
    "dbuser": "***REMOVED SENSITIVE VALUE***",
    "dbpassword": "***REMOVED SENSITIVE VALUE***",
    "installed": true,
    "instanceid": "***REMOVED SENSITIVE VALUE***",
    "check_data_directory_permissions": true,
    "maintenance": false,
    "loglevel": 2,
    "log_type": "file",
    "logfile": "\/var\/www\/html\/data\/nextcloud.log",
    "log_rotate_size": "10485760",
    "log.condition": {
        "apps": [
            "admin_audit"
        ]
    },
    "preview_max_x": "2048",
    "preview_max_y": "2048",
    "jpeg_quality": "60",
    "enabledPreviewProviders": {
        "1": "OC\\Preview\\Image",
        "2": "OC\\Preview\\MarkDown",
        "3": "OC\\Preview\\MP3",
        "4": "OC\\Preview\\TXT",
        "5": "OC\\Preview\\OpenDocument",
        "6": "OC\\Preview\\Movie",
        "0": "OC\\Preview\\Imaginary"
    },
    "enable_previews": true,
    "upgrade.disable-web": true,
    "trashbin_retention_obligation": "auto, 30",
    "versions_retention_obligation": "auto, 30",
    "activity_expire_days": "30",
    "simpleSignUpLink.shown": false,
    "share_folder": "\/Shared",
    "one-click-instance": true,
    "one-click-instance.user-limit": 100,
    "one-click-instance.link": "https:\/\/nextcloud.com\/all-in-one\/",
    "htaccess.RewriteBase": "\/",
    "files_external_allow_create_new_local": true,
    "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
    "preview_imaginary_url": "***REMOVED SENSITIVE VALUE***",
    "default_language": "fr",
    "default_locale": "fr_FR",
    "default_phone_region": "FR",
    "mail_sendmailmode": "smtp",
    "mail_from_address": "***REMOVED SENSITIVE VALUE***",
    "mail_domain": "***REMOVED SENSITIVE VALUE***",
    "mail_smtpauthtype": "LOGIN",
    "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
    "mail_smtpport": "465",
    "mail_smtpauth": 1,
    "mail_smtpname": "***REMOVED SENSITIVE VALUE***",
    "mail_smtppassword": "***REMOVED SENSITIVE VALUE***",
    "allow_local_remote_servers": true,
    "updatedirectory": "\/nc-updater",
    "overwritehost": "notre.rez0.net",
    "overwrite.cli.url": "https:\/\/notre.rez0.net\/",
    "updater.release.channel": "stable",
    "mail_smtpmode": "smtp",
    "mail_smtpsecure": "ssl",
    "upgrade.cli-upgrade-link": "https:\/\/github.com\/nextcloud\/all-in-one\/discussions\/2726",
    "davstorage.request_timeout": 3600,
    "dbpersistent": false,
    "appsallowlist": false,
    "maintenance_window_start": 100,
    "preview_imaginary_key": "***REMOVED SENSITIVE VALUE***",
    "defaultapp": "",
    "auth.bruteforce.protection.enabled": true,
    "ratelimit.protection.enabled": true
}
</details>

Cron Configuration: Array
(
[backgroundjobs_mode] => cron
[lastcron] => 1725993214
)

External storages: files_external is disabled

Encryption: no

User-backends:

OC\User\Database

Talk configuration:

STUN servers

185.252.235.96:443

TURN servers

turn:185.252.235.96:3478 - udp,tcp

Signaling servers (mode: default):

SIP dialin is disabled

SIP dialout is disabled

https://notre.rez0.net/standalone-signaling/ - 1.3.2~docker

Recording servers:

Recording is enabled

Recording consent is set to "default"

no recording server configured

Browser: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/128.0.0.0 Safari/537.36
brakhane commented 2 months ago

I stumbled upon a similiar issue and just post my findings here instead of making a new bug report.

The issue is that postgres does not allow NUL bytes in TEXT fields, however, CustomPropertiesBackend uses serialize to store non scalar values. Sabre's ResourceType has a private field in it, and serialize documentation states:

Object's private members have the class name prepended to the member name; protected members have a '*' prepended to the member name. These prepended values have null bytes on either side.

Also,

Note that this is a binary string which may include null bytes, and needs to be stored and handled as such. For example, serialize() output should generally be stored in a BLOB field in a database, rather than a CHAR or TEXT field.

The problem now is that we try to store that serialized value as-is, which Postgres doesn't support. One way to fix it would be to store non-text values base64 encoded, or alternatively, change the type of propertyvalue from TEXT to BLOB (or BYTEA) and store normal text as UTF-8 encoded blob and objects as-is.

Obviously this would require a database migration.

joshtrichards commented 2 months ago

Also see https://github.com/nextcloud/server/issues/37754#issuecomment-1613361252

punkyard commented 1 month ago

hi @joshtrichards thanks for your reply

I have looked at #37754 and it seems to be about column type when this issue here could be considered as a quantity error what do you think?

starlingfire commented 1 month ago

Also experiencing this issue. I'm running Nextcloud Hub 9 (30.0.0) though the docker image Nextcloud AIO v9.6.0.

I see the below message spammed a ton of times in my log, right after I added my subscribed to my Nextcloud calendar via URL to a Google calendar.

[PHP] Error: unserialize(): Error at offset 61 of 64 bytes at /var/www/html/apps/dav/lib/DAV/CustomPropertiesBackend.php#560
    PROPFIND /remote.php/dav/calendars/[USER_REDACTED]/
    from [IP_REDACTED] by [USER_REDACTED] at Oct 10, 2024, 7:13:59 PM
phil-lipp commented 1 month ago

Also experiencing this issue. I'm running Nextcloud Hub 9 (30.0.0) though the docker image Nextcloud AIO v9.6.0.

I see the below message spammed a ton of times in my log, right after I added my subscribed to my Nextcloud calendar via URL to a Google calendar.

[PHP] Error: unserialize(): Error at offset 61 of 64 bytes at /var/www/html/apps/dav/lib/DAV/CustomPropertiesBackend.php#560
  PROPFIND /remote.php/dav/calendars/[USER_REDACTED]/
  from [IP_REDACTED] by [USER_REDACTED] at Oct 10, 2024, 7:13:59 PM

Same here, also on Hub 9 with Nextcloud AIO. I also have a Google Calendar subscription so that must be it. Calendar entries for that are showing up correctly tho

susnux commented 1 month ago

cc @miaulalala @SebastianKrupinski

SebastianKrupinski commented 1 month ago

Thanks for the heads up @susnux will have a look.

kabatp commented 6 days ago

Can confirm this issue: [PHP] Error: unserialize(): Error at offset 61 of 64 bytes at /var/www/html/apps/dav/lib/DAV/CustomPropertiesBackend.php#560 PROPFIND /remote.php/dav/calendars/[USER]/ from [IP] by [USER] at 20 Nov 2024, 22:48:52

Running Hub9 30.0.2 version with Calendar 5.0.5 on TrueNas Scale 24.10 in docker with postgress

SebastianKrupinski commented 5 days ago

Hi @kabatp is there any more details on this error in the Nextcloud log?

kabatp commented 5 days ago

@SebastianKrupinski all I can find is

{ "reqId": "jt9TMExds0xZthdCzsbl", "level": 3, "time": "2024-11-21T06:39:52+00:00", "remoteAddr": "REDACTED", "user": "REDACTED", "app": "PHP", "method": "PROPFIND", "url": "/remote.php/dav/calendars/REDACTED/", "message": "unserialize(): Error at offset 61 of 64 bytes at /var/www/html/apps/dav/lib/DAV/CustomPropertiesBackend.php#560", "userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:132.0) Gecko/20100101 Firefox/132.0", "version": "30.0.2.2", "data": { "app": "PHP" }, "id": "673ed5b8f05f3" }

As far as I can tell the error started popping up when I tried to set up calendars and contacts. Contacts were manually migrated which means this should not be caused by VCF compatibility issues. Also, I tried to delete all calendars, but the error was still there which I am assuming the problem is with the contacts. The error shows circa every 30 seconds.

The error message points to:

private function decodeValueFromDatabase(string $value, int $valueType) {
                switch ($valueType) {
                        case self::PROPERTY_TYPE_XML:
                                return new Complex($value);
                        case self::PROPERTY_TYPE_HREF:
                                return new Href($value);
                        case self::PROPERTY_TYPE_OBJECT:
                                return unserialize($value);
                        case self::PROPERTY_TYPE_STRING:
                        default:
                                return $value;
                }
        }

specifically to this part

case self::PROPERTY_TYPE_OBJECT:
                                return unserialize($value);

I have tried to play around in the database but couldn't find any data that could cause the issue. Also, this should not be a setup issue as it occurs only with added contacts - when I delete the contacts the issue is no longer there. All of the contacts have been manually created one by one which indicates bad handling of empty fields - as postgress is not good with handling NUL chars as mentioned in https://github.com/nextcloud/server/issues/37754#issuecomment-1589463097

kabatp commented 5 days ago

@SebastianKrupinski I exported contacts from Nextcloud, deleted them and imported them back and it looks like the issue went away. I believe the export contains only fields that are filled which means the issue should be connected to the default fields that are shown in the GUI when creating contacts

kabatp commented 5 days ago

@SebastianKrupinski I exported contacts from Nextcloud, deleted them and imported them back and it looks like the issue went away. I believe the export contains only fields that are filled which means the issue should be connected to the default fields that are shown in the GUI when creating contacts

I am taking back this statement as the issue reappeared after one hour after importing the contacts back