mozilla-services / syncserver

Run-Your-Own Firefox Sync Server
Mozilla Public License 2.0
1.87k stars 145 forks source link

SQLite error with FF for Android #150

Open Groofster opened 6 years ago

Groofster commented 6 years ago

The following error has started happening, even if I let it recreate a new database. Using FF on Android version 62.03

No way of syncing anymore. Using single user mode from the beginning......

(sqlite3.OperationalError) no such column: batch_upload_items.userid [SQL: u"\n INSERT OR REPLACE INTO bso\n (userid, collection, id, sortindex, pay load,\n payload_size, ttl, modified)\n SELECT\n batch_uploads.userid,\n batch_uploads.collection,\n batch_upload_items.id,\n
COALESCE(batch_upload_items.sortindex, existing.sortindex),\n COALESCE(batch_upload_items.payload, existing.payload, ''),\n COALESCE(batch_upload _items.payload_size, existing.payload_size, 0),\n COALESCE(batch_upload_items.ttl_offset + ?, existing.ttl, ?),\n ?\n FROM batch_uploads\n L EFT JOIN batch_upload_items\n ON\n batch_upload_items.batch = batch_uploads.batch AND\n batch_upload_items.userid = batch_uploads.userid\n
LEFT OUTER JOIN bso AS existing\n ON\n existing.userid = batch_uploads.userid AND\n existing.collection = batch_uploads.collection AND\n
existing.id = batch_upload_items.id\n WHERE\n batch_uploads.batch = ? AND\n batch_uploads.userid = ?\n / [queryName=APPLY_BATCH_INSERT] /"] [parameters: (1540476889, 2100000000, 1540476889420, 1540476889421, 1)] (Background on this error at: http://sqlalche.me/e/e3q8)

rfk commented 6 years ago

Thanks for the report. Are you only using an Android device, or do you have other devices that are connecting successfully? Did you recently update your server and if so, from what version?

Groofster commented 6 years ago

Hi, I'm running FF 63.00 64-bit on Ubuntu Linux, FF 63.00 64-bit on Windows 10 and FF 62.03 on multiple Android devices. I have tested sync on all of them and they all give the same error now in the docker logs (sorry, forgot to mention I run the sunx/mozilla-syncserver docker container). In my FF account information it is stated that even the new Android device I added has synced as recently as 8 hours ago which is not a valid statement for that device not doing an initial sync was the reason I stumbled upon the error. Unfortunately that makes it impossible for me to determine when exactly the error started.

I'm sure I've updated to version 1.8 of the docker image in the past couple of days so that might be the culprit.

Any advice ? As stated, backing up the syncserver.db file and letting the container create a new one gives me the exact same error.

/edit missing words

rfk commented 6 years ago

backing up the syncserver.db file and letting the container create a new one gives me the exact same error.

Would you be happy to share the re-created (and I assume empty) syncserver.db with me via email so I can see the table structure etc? My email is discoverable in my github profile.

Groofster commented 6 years ago

More than happy to. It should be in your mail by now. Thanks in advance

rfk commented 6 years ago

no such column: batch_upload_items.userid

Strange, the re-created database file definitely does have this column:

$ sqlite3 ~/Desktop/syncserver.db
SQLite version 3.11.0 2016-02-15 17:29:24
Enter ".help" for usage hints.
sqlite> .schema batch_upload_items
CREATE TABLE batch_upload_items (
        batch BIGINT NOT NULL,
        userid INTEGER NOT NULL,
        id VARCHAR(64) NOT NULL,
        sortindex INTEGER,
        payload TEXT,
        payload_size INTEGER,
        ttl_offset INTEGER,
        PRIMARY KEY (batch, userid, id)
);

(sorry, forgot to mention I run the sunx/mozilla-syncserver docker container)

I tried to reproduce by also running this docker contain, but I wasn't able to - it successfully created a db and allow my firefox to sync.

So...I'm not really sure what to suggest here sorry. I suppose it's possible that some config option has changed since you installed the service, and deleting your syncserver.ini and allowing the docker image to re-create it might help.

You could also try preffing off batch uploads by adding this to your config file:

[storage]
batch_upload_enabled=false

But otherwise I'm out of ideas :-/

Groofster commented 6 years ago

Thank you for your time and effort. I will try and test your suggestion and will let you know the results.

Groofster commented 6 years ago

I have tested your suggestion and that works like a charm. Setting batch_upload_enabled to true (after all my devices had synced without issues) gives the same error again. So there is something wrong with batch syncing (at least on my configuration).

If you have any more questions, suggestions or what not, please let me know. For now I have a working solution.

Again, thanks for your time