Loading each item with HTTP is very time consuming.
By loading the items in a thread the main migration thread does no longer need to wait while loading each item.
The queue only loads a maximum amount of items (default: 10) so that the RAM does not get filled up.
I was able to speed up a small, not representative migration to 30% of the original time.
Loading each item with HTTP is very time consuming. By loading the items in a thread the main migration thread does no longer need to wait while loading each item. The queue only loads a maximum amount of items (default: 10) so that the RAM does not get filled up.
I was able to speed up a small, not representative migration to 30% of the original time.