mozilla-services / syncserver

Run-Your-Own Firefox Sync Server
Mozilla Public License 2.0
1.87k stars 145 forks source link

Missing columns in batch_upload_items when upgrading older syncserver #214

Open new0ne opened 4 years ago

new0ne commented 4 years ago

Hi there! After upgrading my syncserver to the current version it ran into several errors: the users table was missing the columns: keys_changed_at as well as node. Luckily i found https://github.com/mozilla-services/syncserver/issues/208 and applied the suggested changes. But other than that my table batch_upload_items has a different schema than its expected in head, which leads to errors like: InternalError: (pymysql.err.InternalError) (1054, u"Unknown column 'userid' in 'where clause'") [SQL: / [queryName=APPLY_BATCH_INSERT] / INSERT INTO bso (userid, collection, id, modified, sortindex,ttl, payload, payload_size) SELECT %(userid)s, %(collection)s, id, %(modified)s, sortindex, COALESCE(ttl_offset + %(ttl_base)s, %(default_ttl)s), COALESCE(payload, ''), COALESCE(payload_size, 0) FROM batch_upload_items WHERE batch = %(batch)s AND userid = %(userid)s ON DUPLICATE KEY UPDATE modified = %(modified)s, sortindex = COALESCE(batch_upload_items.sortindex, bso.sortindex), ttl = COALESCE(batch_upload_items.ttl_offset + %(ttl_base)s, bso.ttl), payload = COALESCE(batch_upload_items.payload, bso.payload), payload_size = COALESCE(batch_upload_items.payload_size, bso.payload_size) ... my table batch_upload_items looks like: +--------------+-------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +--------------+-------------+------+-----+---------+-------+ | batch | bigint(20) | NO | PRI | NULL | | | id | varchar(64) | NO | PRI | NULL | | | sortindex | int(11) | YES | | NULL | | | payload | mediumtext | YES | | NULL | | | payload_size | int(11) | YES | | NULL | | | ttl_offset | int(11) | YES | | NULL | | +--------------+-------------+------+-----+---------+-------+

I am trying to find the point in time (in the repo) where that got changed and finding a "database migration script" if there is any. Would be awesome if somebody knows how to fix that.

Kind regards

rfk commented 4 years ago

finding a "database migration script" if there is any

Unfortunately we haven't built one of those for this repo. In this case I think your best bet is to drop the batch_uploads and batch_upload_items items tables, restart the server and let is re-create them with correct schema. These tables are only used for temporary data so this should be a safe operation.