Closed lastephey closed 4 years ago
Rahul reminded me that pinning memory can be very expensive. It only pays off if you are going to use it many times.
In the case that we accumulate data and transfer one time (i.e. per bundle), we most likely wouldn't want to pin memory.
We are now transferring at the end of each bundle which seems fine and has a low memory transfer overhead
One question that arose during the hackathon was whether it made sense to transfer data from each patch back the the cpu every time or to accumulate patch data and transfer one time after the entire bundle is complete.
Explore how much performance gain could come from accumulating data for a single DtH transfer rather than many smaller transfers.
However, weigh this against the amount of data we must keep in memory as we accumulate results from each patch. Determine if this memory accumulation is feasible.
Compare additional memory requirements to DtH performance wins to recommend memory transfer strategy (i.e. per patch or per bundle).