Client 1.6.1 on Windows XP sends corrupted xlt file

[brozkeff]

Expected behaviour

When I create/edit an Excel (or ANY OTHER) file in a folder shared with Owncloud it always uploads the file correctly and does not upload and spread a corrupted version of such file

Actual behaviour

Randomly (I don't know what exactly caused it) I experienced that a XLT template modified by Excel 2002 on a WinXP machine got garbled in Owncloud and was downloaded immediately by another computer login to same account (Linux Mint 13 MATE 1.8, 32bit, OC client 1.6.0) and also to a colleague (Win7, OC client 1.6.1) that has that folder shared (read-only). Instead of actual Excel template the file was a garbage unreadable both by Excel 2010 and LibreOffice. The filesize was OK. (176 128 bytes) Deleting the file from the "master" (WinXP), waiting till OC client deleted it on server and other clients, and copying the file there again, eventually led to correct synchronization.

Steps to reproduce

Seems random, cannot guarantee it happens always.

1. create Excel 2002 xlt file (size 176 128b), move it from Templates to OC shared directory
2. wait till it is uploaded to OC server
3. sometimes the file is transmitted as garbage

   Server configuration

Operating system: Ubuntu 14.04 server in a OpenVZ container

Web server: Apache 2.4.7-1ubuntu4

Database: MySQL 5.5.35+dfsg-1ubuntu1

PHP version: 5.5.9+dfsg-1ubuntu4

ownCloud version: 7.0RC1

Updated from an older ownCloud or fresh install: update via apt-get from 6.0.4

List of activated apps: Activity Deleted files Documents External user support Owncloud dep.info Pictures Share files Text editor Updater Versions Video viewer

Client configuration

Client version: 1.6.1 on Windows XP where I created the XLT files 1.6.0 on Linux Mint 13 32bit (connected to same account as the WinXP machine) 1.6.1 on Win7 64bit (connected to another account but had that folder with the XLT files shared readonly)

Operating system: WIndows XP Home 32bit SP3

OS language: Czech

Installation path of client:

Logs

I had debug on and there are now hundreds of MB of logs and I don't have time now to parse it , remove all confidential info etc. If you need something tell me which strings to look for.

[ogoffart]

Where does the corruption happen? While transmitting the file to the server or is th efile fine on the server and transmiting back to other client creates the corruption? Is the file being modified durring the upload? Is the file still in good in the original computer? And if the file size is the same, what are the differences? Would you be able to try to do a binary diff? Was there any warning on error on the clients?

[brozkeff]

The file is saved as corrupted already on the server - By logging to web interface I was able to obtain one of the corrupted versions of the file. The first part of the file seems to me as binary nonsense.

The original client seemed to keep the original (correct) file but as soon as I recognized the bug I backuped these files to a non-synced location, because I needed to delete the files from OC and then re-copy them.

No warning error on the clients, and in their logs. Just messages of repeated downloading the same file.

I send you the correct and corrupted versions of one of these files to email, because the file contains company data I don't want to share publicly on github.

[ogoffart]

Both fthe file you sent me seems to be corrupted. I can open both in libreoffice, but they just contains many pages of garbage.

The file are completely different at the begining (i could not find a pattern). then from the range 0x4000 to the end, they are mostly equal with some small differences here and there and a big chunk that is with an offset.

I don't know how such corruption can appear. Maybe the file was modified when it was sent and part of an old file were sent?

[brozkeff]

I rechecked the files I sent you, the "ok"
  should be OK and opens correctly on my Libreoffice 4.2.5.2
even when I open it from the attachment of the sent mail.
If files get uploaded partially overwritten by newer version then
such behaviour must be considered as a serious bug - Owncloud can't
just pretend silently the uploaded file is OK when in fact it is
not. Some checksum validation should be performed perhaps,and try to
re-upload such corrupted file if it occurs?

[brozkeff]

I am still experiencing it when using Windows. It seems to occus also on Windows 8.1 where mere opening of an InDesign file in AI CS 6 from owncloud-synced folder caused that file to get reuploaded to the server in a corrupted way.

My colleagues got also corrupted a regular excel file after doing some edits on it, they use Windows 7 Home premium.

I did not ever experience this behaviour from my Linux Mint that I use most of the time. When using Adobe Creative Suite, I need however to use Windows and then I try to close owncloud completely, then do all the work with files in Illustrator or Indesign or Excel, and run owncloud client manually back after I finish, and then not touching it until it is reuploaded.

[brozkeff]

I consider Issue #2425 to indeed be duplicate of this issue.

And I still suffer from this problem. Random files but mostly those I work get corrupted in the beginning. THe timestamp and filesize are the same so even after I copy the correct file back from staging to the owncloud direktory the client refuses to upload it as a new version. Unless this issue can be solved (so no corruption happens at all) I think that it is urgently needed to make also regular checksums and compare them. Because to me it seems the problem always occurs at the beginning of the file it would be perhaps possible to do a hash only from the first 64kB and the last 64kB data, so both this corruption from the beginning as well as at the end (https://github.com/owncloud/mirall/issues/2210 #2210) could be detected.

[ogoffart]

I'm closing this issue and we can continue the discussion on #2425

BTW, maybe 1.8 is better because we read the chunk in memory before sending them.