Closed temas closed 12 years ago
+1 -- have you done any measurement of how this synclet affects the core's memory usage when run on a vm?
+1, looks good, also to note that everything will have to page and hammer a bit at least once more, then calm after that.
Core with master, no VM, multiple processes required, etc completed run: { rss: 41803776, heapTotal: 29588864, heapUsed: 15668432 }
Core with VM version after completed run: { rss: 28688384, heapTotal: 17844352, heapUsed: 12853896 }
It's also overall faster as it uses a larger paging size and does not incur a multiple process penalty.
This is a shortest path attempt to clean up the flickr synclet from it's previous full scan state. It now uses the upload time to decide what's new and upped the per page batch size to 500.