search

nextcloud / richdocuments

📑 Collabora Online for Nextcloud
https://nextcloud.com/collaboraonline
355 stars 116 forks source link

[Bug]: Office files are corrupted/emptied after some time #3289

Open cyberb opened 1 year ago

cyberb commented 1 year ago

⚠️ This issue respects the following points: ⚠️

Bug description

I use Collabora Online server on the same box as Nextcloud to edit libre calc ods files and it works great I can edit a file on web and then on mobile phone. But some times I come back to the file and it is empty, I check versions and they exist but they are all also empty no matter what revision I open. This happened twice with different versions of Nextcloud, I lost hours of work :( I will have to make a copy manually and not open it using Nextcloud Office.

Steps to reproduce

  1. Create Calc file (ods)
  2. Edit on desktop browser
  3. Edit on mobile phone
  4. After some time open on desktop web and it is empty
  5. Open on mobile phone it is empty
  6. Check revision history all versions are empty (you can see it from their sizes)

Expected behavior

If some device corrupts the file it should at least not corrupt all the versions

Installation method

Other Community project

Nextcloud Server version

27

Operating system

Debian/Ubuntu

PHP engine version

PHP 8.2

Web server

Nginx

Database engine version

PostgreSQL

Is this bug present after an update or on a fresh install?

None

Are you using the Nextcloud Server Encryption module?

None

What user-backends are you using?

Configuration report

{
    "system": {
        "datadirectory": "***REMOVED SENSITIVE VALUE***",
        "log_type": "syslog",
        "logfile": "\/var\/snap\/nextcloud\/common\/log\/nextcloud.log",
        "apps_paths": [
            {
                "path": "\/snap\/nextcloud\/current\/nextcloud\/apps",
                "url": "\/apps",
                "writable": false
            },
            {
                "path": "\/var\/snap\/nextcloud\/current\/extra-apps",
                "url": "\/extra-apps",
                "writable": true
            }
        ],
        "passwordsalt": "***REMOVED SENSITIVE VALUE***",
        "secret": "***REMOVED SENSITIVE VALUE***",
        "trusted_domains": [
            "localhost",
            "192.168.1.***",
            "nextcloud.***.syncloud.it"
        ],
        "dbtype": "pgsql",
        "version": "27.1.3.2",
        "overwrite.cli.url": "http:\/\/localhost",
        "dbname": "***REMOVED SENSITIVE VALUE***",
        "dbhost": "***REMOVED SENSITIVE VALUE***",
        "dbport": "",
        "dbtableprefix": "oc_",
        "dbuser": "***REMOVED SENSITIVE VALUE***",
        "dbpassword": "***REMOVED SENSITIVE VALUE***",
        "installed": true,
        "ldapIgnoreNamingRules": false,
        "ldapProviderFactory": "OCA\\User_LDAP\\LDAPProviderFactory",
        "memcache.local": "\\OC\\Memcache\\APCu",
        "loglevel": "2",
        "mail_smtpmode": "smtp",
        "mail_smtphost": "***REMOVED SENSITIVE VALUE***",
        "instanceid": "***REMOVED SENSITIVE VALUE***",
        "maintenance": false,
        "trusted_proxies": "***REMOVED SENSITIVE VALUE***",
        "default_phone_region": "GB",
        "mail_domain": "***REMOVED SENSITIVE VALUE***",
        "mail_sendmailmode": "smtp",
        "mail_from_address": "***REMOVED SENSITIVE VALUE***",
        "mail_smtpauth": false,
        "check_data_directory_permissions": false
    }
}

List of activated Apps

Enabled:
  - activity: 2.19.0
  - circles: 27.0.1
  - cloud_federation_api: 1.10.0
  - comments: 1.17.0
  - contactsinteraction: 1.8.0
  - dashboard: 7.7.0
  - dav: 1.27.0
  - federatedfilesharing: 1.17.0
  - federation: 1.17.0
  - files: 1.22.0
  - files_pdfviewer: 2.8.0
  - files_reminders: 1.0.0
  - files_rightclick: 1.6.0
  - files_sharing: 1.19.0
  - files_trashbin: 1.17.0
  - files_versions: 1.20.0
  - firstrunwizard: 2.16.0
  - logreader: 2.12.0
  - lookup_server_connector: 1.15.0
  - music: 1.9.1
  - nextcloud_announcements: 1.16.0
  - notifications: 2.15.0
  - oauth2: 1.15.1
  - password_policy: 1.17.0
  - photos: 2.3.0
  - privacy: 1.11.0
  - provisioning_api: 1.17.0
  - recommendations: 1.6.0
  - related_resources: 1.2.0
  - richdocuments: 8.2.2
  - serverinfo: 1.17.0
  - settings: 1.9.0
  - sharebymail: 1.17.0
  - spreed: 17.1.2
  - support: 1.10.0
  - survey_client: 1.15.0
  - systemtags: 1.17.0
  - text: 3.8.0
  - theming: 2.2.0
  - twofactor_backupcodes: 1.16.0
  - user_ldap: 1.17.0
  - user_status: 1.7.0
  - viewer: 2.1.0
  - weather_status: 1.7.0
  - workflowengine: 2.9.0
Disabled:
  - admin_audit: 1.17.0
  - bruteforcesettings: 2.7.0
  - encryption: 2.15.0
  - files_external: 1.19.0
  - maps: 1.1.0 (installed 1.1.0)
  - suspicious_login: 5.0.0
  - twofactor_totp: 9.0.0

Nextcloud Signing status

No errors have been found.

Nextcloud Logs

No response

Additional info

No response

joshtrichards commented 1 year ago

Can you look in your Nextcloud server logs for entries mentioning one of these corrupted files? Both entries mentioning the file and immediately before it would be of interest.

Also, what external Collabora Online version/edition are you using?

Lastly, to clarify, you are able to edit in one place (i.e. web) the same file... Then go to your phone's Nextcloud client and edit the file successfully? But the. later the file disappears? Does it ever disappear prior to accessing from a second device?

cyberb commented 12 months ago

Sorry I do not have logs now, but when I checked both nextcloud and collabora logs they had no exceptions and looked similar to other files.

Also, what external Collabora Online version/edition are you using?

I am actually extracting binaries from their docker image: collabora/code:21.11.4.2.1 And then I convert it to a snap package: https://github.com/syncloud/collabora/blob/master/app/Dockerfile

  1. I am able edit on the web and collabora web opens. Auto saves and I leave it running.
  2. Then continue adding more stuff on the mobile. Now as I opened the file again actually it is collabora mobile app
  3. Then opened on the web again and it was empty.

I can try to reproduce it to give more details, maybe it is related to user session exporation or something else.

cyberb commented 12 months ago

Ok just tried to repro and it happened again!

  1. Create a new file on Nextcloud Android (test1.ods) embedded collabora calc opens (looks like embedded web one, not my collabora mobile app) add few test entries. Save. Not close.
  2. Open nextcloud we in firefox same mobile phone. File exists but empty.
  3. Go back to Nextcloud mobile app with open file, not it says taking too long to open (it was actually open, but I guess app switch causes to reload file). Click cancel, reopen file again and it is empty now.

Logs from Nextcloud server:

root@syncloud:~# journalctl -u snap.nextcloud.* | grep test1.ods
Nov 11 08:41:53 syncloud nginx[2915]: syncloud nginx: *** - boris [11/Nov/2023:08:41:53 +0000] "PROPFIND /remote.php/dav/files/boris/Temp/test1.ods HTTP/1.1" 207 1308 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.26.0"
Nov 11 08:43:28 syncloud nginx[2914]: syncloud nginx: ***  - - [11/Nov/2023:08:43:28 +0000] "GET /apps/files/?dir=/Temp&openfile=1322457&scrollto=test1.ods HTTP/1.1" 200 16904 "-" "Mozilla/5.0 (Android 10; Mobile; rv:109.0) Gecko/119.0 Firefox/119.0"
Nov 11 08:43:33 syncloud nginx[2914]: syncloud nginx: ***  - - [11/Nov/2023:08:43:33 +0000] "GET /ocs/v2.php/search/providers?from=%2Fapps%2Ffiles%2F%3Fdir%3D%2FTemp%26openfile%3D1322457%26scrollto%3Dtest1.ods HTTP/1.1" 200 277 "-" "Mozilla/5.0 (Android 10; Mobile; rv:109.0) Gecko/119.0 Firefox/119.0"
Nov 11 08:43:38 syncloud nginx[2915]: syncloud nginx: ***  - - [11/Nov/2023:08:43:38 +0000] "PROPFIND /remote.php/dav/files/boris/Temp/test1.ods HTTP/1.1" 207 585 "-" "Mozilla/5.0 (Android 10; Mobile; rv:109.0) Gecko/119.0 Firefox/119.0"
Nov 11 08:44:22 syncloud nginx[2915]: syncloud nginx: ***  - boris [11/Nov/2023:08:44:22 +0000] "GET /index.php/apps/files/api/v1/thumbnail/256/256/Temp/test1.ods HTTP/1.1" 404 29 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.26.0"

Collabora server logs:

journalctl -u snap.collabora.* | grep test1.ods
Nov 11 08:41:57 syncloud collabora.server[497]: wsd-01320-02476 2023-11-11 08:41:57.706792 +0000 [ websrv_poll ] INF  nextcloud/server#21: Client HTTP Request: POST /browser/3eff190/cool.html?WOPISrc=https%3A%2F%2Fnextcloud.***.syncloud.it%2Findex.php%2Fapps%2Frichdocuments%2Fwopi%2Ffiles%2F1322457_ocy0e4tzvl95&title=test1.ods&lang=en-GB&closebutton=1 HTTP/1.0 / Host: collabora.***.syncloud.it / Connection: close / Content-Length: 4246 / X-Forwarded-Proto: https / X-Forwarded-Host: collabora.***.syncloud.it / X-Real-IP: *** / X-Forwarded-For: *** / cache-control: max-age=0 / sec-ch-ua: "Chromium";v="118", "Android WebView";v="118", "Not=A?Brand";v="99" / sec-ch-ua-mobile: ?1 / sec-ch-ua-platform: "Android" / upgrade-insecure-requests: 1 / origin: null / content-type: application/x-www-form-urlencoded / user-agent: Mozilla/5.0 (Android) Nextcloud-android/3.26.0 / accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7 / x-requested-with: com.nextcloud.client / sec-fetch-site: same-site / sec-fetch-mode: navigate / sec-fetch-dest: iframe / accept-encoding: gzip, deflate, br / accept-language: en-GB,en-US;q=0.9,en;q=0.8| net/Socket.cpp:1071
Nov 11 08:41:57 syncloud nginx[1388]: syncloud nginx: *** - - [11/Nov/2023:08:41:57 +0000] "POST /browser/3eff190/cool.html?WOPISrc=https%3A%2F%2Fnextcloud.***.syncloud.it%2Findex.php%2Fapps%2Frichdocuments%2Fwopi%2Ffiles%2F1322457_ocy0e4tzvl95&title=test1.ods&lang=en-GB&closebutton=1 HTTP/1.1" 200 31706 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.26.0"
Nov 11 08:41:58 syncloud collabora.server[497]: kit-13368-01946 2023-11-11 08:41:58.305061 +0000 [ kitbroker_00a ] INF  Saving the template document after loading to [file:///var/snap/collabora/current/child-roots/vxu5YoKVNakdaJVc//var/snap/collabora/current/child-roots/vxu5YoKVNakdaJVc/tmp/user/docs/vxu5YoKVNakdaJVc/test1.ods].| kit/ChildSession.cpp:741
Nov 11 08:41:58 syncloud collabora.server[497]: wsd-01320-01325 2023-11-11 08:41:58.607738 +0000 [ docbroker_00a ] ERR  Expected to renamed the document [/var/snap/collabora/current/child-roots/vxu5YoKVNakdaJVc/tmp/user/docs/vxu5YoKVNakdaJVc/test1.ods.upload] after template-loading to [/var/snap/collabora/current/child-roots/vxu5YoKVNakdaJVc/tmp/user/docs/vxu5YoKVNakdaJVc/test1.ods.uploading] (ENOENT: No such file or directory)| wsd/DocumentBroker.cpp:1379
Nov 11 08:41:59 syncloud collabora.server[497]: wsd-01320-01325 2023-11-11 08:41:59.338410 +0000 [ docbroker_00a ] ERR  Expected to renamed the document [/var/snap/collabora/current/child-roots/vxu5YoKVNakdaJVc/tmp/user/docs/vxu5YoKVNakdaJVc/test1.ods.upload] after template-loading to [/var/snap/collabora/current/child-roots/vxu5YoKVNakdaJVc/tmp/user/docs/vxu5YoKVNakdaJVc/test1.ods.uploading] (ENOENT: No such file or directory)| wsd/DocumentBroker.cpp:1379
Nov 11 08:43:38 syncloud collabora.server[497]: wsd-01320-02476 2023-11-11 08:43:38.756012 +0000 [ websrv_poll ] INF  nextcloud/server#21: Client HTTP Request: POST /browser/3eff190/cool.html?WOPISrc=https%3A%2F%2Fnextcloud.***.syncloud.it%2Findex.php%2Fapps%2Frichdocuments%2Fwopi%2Ffiles%2F1322457_ocy0e4tzvl95&title=%2FTemp%2Ftest1.ods&lang=en-GB&closebutton=1&revisionhistory=1 HTTP/1.0 / Host: collabora.***.syncloud.it / Connection: close / Content-Length: 4233 / X-Forwarded-Proto: https / X-Forwarded-Host: collabora.***.syncloud.it / X-Real-IP: *** / X-Forwarded-For: *** / user-agent: Mozilla/5.0 (Android 10; Mobile; rv:109.0) Gecko/119.0 Firefox/119.0 / accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,*/*;q=0.8 / accept-language: en-GB / accept-encoding: gzip, deflate, br / content-type: application/x-www-form-urlencoded / origin: null / upgrade-insecure-requests: 1 / sec-fetch-dest: iframe / sec-fetch-mode: navigate / sec-fetch-site: same-site| net/Socket.cpp:1071
Nov 11 08:43:38 syncloud nginx[1391]: syncloud nginx: *** - - [11/Nov/2023:08:43:38 +0000] "POST /browser/3eff190/cool.html?WOPISrc=https%3A%2F%2Fnextcloud.***.syncloud.it%2Findex.php%2Fapps%2Frichdocuments%2Fwopi%2Ffiles%2F1322457_ocy0e4tzvl95&title=%2FTemp%2Ftest1.ods&lang=en-GB&closebutton=1&revisionhistory=1 HTTP/1.1" 200 31694 "-" "Mozilla/5.0 (Android 10; Mobile; rv:109.0) Gecko/119.0 Firefox/119.0"
Nov 11 08:44:28 syncloud collabora.server[497]: wsd-01320-02476 2023-11-11 08:44:28.980647 +0000 [ websrv_poll ] INF  nextcloud/server#21: Client HTTP Request: POST /browser/3eff190/cool.html?WOPISrc=https%3A%2F%2Fnextcloud.***.syncloud.it%2Findex.php%2Fapps%2Frichdocuments%2Fwopi%2Ffiles%2F1322457_ocy0e4tzvl95&title=test1.ods&lang=en-GB&closebutton=1 HTTP/1.0 / Host: collabora.***.syncloud.it / Connection: close / Content-Length: 4246 / X-Forwarded-Proto: https / X-Forwarded-Host: collabora.***.syncloud.it / X-Real-IP: *** / X-Forwarded-For: *** / cache-control: max-age=0 / sec-ch-ua: "Chromium";v="118", "Android WebView";v="118", "Not=A?Brand";v="99" / sec-ch-ua-mobile: ?1 / sec-ch-ua-platform: "Android" / upgrade-insecure-requests: 1 / origin: null / content-type: application/x-www-form-urlencoded / user-agent: Mozilla/5.0 (Android) Nextcloud-android/3.26.0 / accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7 / x-requested-with: com.nextcloud.client / sec-fetch-site: same-site / sec-fetch-mode: navigate / sec-fetch-dest: iframe / accept-encoding: gzip, deflate, br / accept-language: en-GB,en-US;q=0.9,en;q=0.8| net/Socket.cpp:1071
Nov 11 08:44:28 syncloud nginx[1391]: syncloud nginx: ***  - - [11/Nov/2023:08:44:28 +0000] "POST /browser/3eff190/cool.html?WOPISrc=https%3A%2F%2Fnextcloud.***.syncloud.it%2Findex.php%2Fapps%2Frichdocuments%2Fwopi%2Ffiles%2F1322457_ocy0e4tzvl95&title=test1.ods&lang=en-GB&closebutton=1 HTTP/1.1" 200 31706 "-" "Mozilla/5.0 (Android) Nextcloud-android/3.26.0"
cyberb commented 12 months ago

All versions are empty now. IMG_20231111_085622

joshtrichards commented 12 months ago

Logs from Nextcloud server:

Those are all helpful, but these logs are from your web server (Nginx) not Nextcloud Server. Can you share those from the same time period? Thanks.

Oh also since some things (like errors) may not be logged with the user filename, can you share the Collabora logs for that time window rather than just grep'd by filename?

Also going to move this Issue over to the richdocuments repository for now.

cyberb commented 12 months ago

Just reproduced again using the same workflow, there is nothing from nextcloud itself, usually I have it in the same output from systemd as I run it using snap.nextcloud.phpfpm process.

Also I have this in the config:

 'log_type' => 'syslog',
cyberb commented 12 months ago

I can try upgrading collabora to 23.05.5.3.1 to see if that helps, at least I can reproduce it now.

cyberb commented 12 months ago

Ok, upgrade to 23.05.5.3.1 actually helped at least now I cannot reproduce the isssue, I will reopen if it happens again.

cyberb commented 12 months ago

Sorry the problem is there still. Yesterday I eas testing a file and was switching between mobile app and web collabora integrations, eberything was fine and changes instantly visible. Today opened the same file on the web and it is empty and history shows empty changes.

Screenshot_2023-11-15-09-32-33-661_com nextcloud client

cyberb commented 12 months ago

I will try to get more logs

marcinkuk commented 10 months ago

I heve exacltly the same issue with my nextclouds. Have several nextcloud environements with different version and collabora or nextcloud sometimes eat my files. I lost a lot of work.

pehlert commented 9 months ago

Same here, ods file completely empty after a while, showing the same size (683 B) for all versions.

Needless to say this is disastrous for a software who's main purpose is so safely store files.

voggur commented 9 months ago

Same bug as: Document content is not saved when it was just created ?

artfulrobot commented 9 months ago

This is a terrifying bug! Glad I found this report in time, I was going to recommend nextcloud office to a client, but I would be pilloried if this happened to their docs! As I understand, Nextcloud passes the file to CODE which then somehow passes it back after an update, which makes sense, but blanking all the previous versions?! I don't get that at all.

Not sure if it's related (and will check the issue queue) but I get similarly weird behaviours with Excalidraw - like content is just not there, or is old etc. Figured it could be related as it's another case of pass the doc out somewhere and store it when updated.

zsQ8ymRSY8B4owpdCvTRdkz8rQUK commented 6 months ago

+1. Just had a client lose all of their work in an ods after they edited the file on their phone app. All versions became empty.

Has it been acknowledged if this is being worked on?

estux commented 6 months ago

It seems there's a link with mobile editing (with Collabora app) and it could be also a Collabora problem. Did someone opened an issue also on their side?

estux commented 6 months ago

It seems there's a link with mobile editing (with Collabora app) and it could be also a Collabora problem. Did someone opened an issue also on their side?

Replying to myself: Linked to this bug https://github.com/CollaboraOnline/online/issues/8242 in the Collabora Online repo.

ServError commented 5 months ago

Experienced this bug myself yesterday after installing built in CODE and migrating some financial documents. I actually couldn't find this issue at the time and stumbled upon only by accident today, so maybe it needs more tags. The data loss in previous versions was mortifying and made me look incompetent. It's disappointing to see that it was reported 7 months ago.

In my instance we were also playing with opening the document from the Nextcloud android app after creating and editing the file from the desktop browser.

estux commented 4 months ago

I'm just writing here to add that something about this seems it was fixed with the new release of Collabora CODE: https://github.com/CollaboraOnline/richdocumentscode/issues/238#issuecomment-2182555151 I hope at least this solves some of your problems.

poiNt3D commented 1 month ago

Hi. I'm experiencing a similar issue, but only current version is overwritten with empty (0 kb). Still can't get the exact stemps, but i'm just fiddeling around: scrolling up and down, adjusting my router settings, refreshing, etc. No real editing. But sometimes i get an empty file somehow.

cyberb commented 1 month ago

What version of CODE are you on?

bartart3d commented 1 month ago

Using nextcloud office Version 8.4.7 on nextcloud 29.0.7

cyberb commented 1 month ago

People say it is fixed in Collabora (CODE) 24.4.402. What version do you have? CODE is not nextcloud or nextcloud office app.

bartart3d commented 4 weeks ago

hello, My installed version was 24.4.702, now about to upgrade to 24.4.802