search

protocol / beyond-bitswap

Other
34 stars 9 forks source link

The testbed can't generate random files over 100MB #1

Closed adlrocha closed 3 years ago

adlrocha commented 3 years ago

For the option to use random files in the testbed, the local machine chokes when trying to generate files over 100MB. This issue may extrapolate to cloud environments because in the current implementation a single seed generates the full file. We should devise a way in which all seeds collaborate on generating non overlapping pieces of the file so that the single seed doesn't choke.

adlrocha commented 3 years ago

Randomly generating files is computationally expensive, and it is not possible inside experiments where there is a hardware limitation. A workaround to this is proposed in 5402f3fb3e1efe52b4e172daa0c1755fbdbeb5c0. Random files should be prepared and generated beforehand for the experiment using testbed/scripts/random-file.sh.