golemfactory / gumpi

MPI on Golem Unlimited PoC
GNU Affero General Public License v3.0
2 stars 3 forks source link

During upload, read the file as an asynchronous stream #44

Open marmistrz opened 5 years ago

marmistrz commented 5 years ago

Currently we just synchronously read the file all at once.

This would be useful, since the input data may be large, and the computer where gumpi is running may have much less RAM than the provider nodes.

Introduced by #40.