coresmart / persistencejs

persistence.js is an asynchronous Javascript database mapper library. You can use it in the browser, as well on the server (and you can share data models between them).
http://persistencejs.org
1.73k stars 240 forks source link

Syncing down thousands of changes at a time doesn't work #99

Closed pixelcort closed 11 years ago

pixelcort commented 11 years ago

I've got sync set up and working, however I've got thousands of changes that were made server side.

After sync calls GET /api/testSync?since=1344020688591 and gets back the huge changes array, it tries to do a local DB query:

SELECT `root`.id AS Deck_id, `root`.`name` AS `Deck_name`, `root`.`lastUploaded` AS `Deck_lastUploaded`, `root`.`_lastChange` AS `Deck__lastChange` FROM `Deck` AS `root`  WHERE (1=1 AND `root`.`id` IN (?, ?, ?, ?, ?, ?, <snip>

... where the IN clause has thousands of arguments.

I'm betting the issue is that, in Chrome and probably Safari, it appears there is a limit to the number of arguments to an IN clause.

As a workaround, I considered only sending to the client a few records at a time with a forged now key, and then just run sync over and over again, however that won't work in this case since the thousands of changes all have the exact same _lastChange key.