Test case 1st refresh creating ~1.2M records, 2nd refresh updating ~1.2M records with no changes, with 3 persisters and 5 partitions (with this setup, 1 persister is slacking, since it get only 1 partitions assigned)
with use_ar_object => true
refresh
max saving 1 part
avg saving 1 part
total time
max memory
1st
1.53s
0.47s
28.5m
100MB
2nd
1.09s
0.38s
25.3m
105MB
with use_ar_object => false
refresh
max saving 1 part
avg saving 1 part
total time
max memory
1st
1.53s
0.47s
28.5m
100MB
2nd
1.06s
0.28s
25.15m
97MB
Evaluation
use_ar_object => false is a bit quicker, but it always updates everything, so it stresses the DB much more. Also by updating everything, we don't get reliable report about what was changed.
Test case 1st refresh creating ~1.2M records, 2nd refresh updating ~1.2M records with no changes, with 3 persisters and 5 partitions (with this setup, 1 persister is slacking, since it get only 1 partitions assigned)
with use_ar_object => true
with use_ar_object => false
Evaluation
use_ar_object => false is a bit quicker, but it always updates everything, so it stresses the DB much more. Also by updating everything, we don't get reliable report about what was changed.