Wishlist 5.x - Chunk Prefetching / Multi-Fetching

[lovesyk]

Let's say we have 3 chunkservers. One on a GBit connection, one on 500 MBit and one on 100 MBit. There is no replication set up and only one client is constantly consuming data at 150 MBit/s. Currently, chunks are being fetched sequentially as they are requested. This means that the 100 MBit server will be overloaded once its turn comes up and the client will slow down to 100 Mbit/s until the current chunk is done. After that, the 100 MBit server is going to idle until the next two chunks have been consumed from the other servers, then get overloaded again. Wouldn't it be great if the LizardFS client was more intelligent and recognized that:

• the user requested the first chunk of a huge file so he is likely to request the following chunks as well
• according to previous transfers, there is one very fast server in the network, another slower one and yet another very slow one and those are the best places (well, the only places) it can get the chunks of that file from
• the 100 MBit chunkserver will most likely be the bottleneck of the current operation and its chunks should be fetched in advance with a lower priority
• prefetching chunks of the 500 MBit server should have even lower priority and those of the GBit one the lowest

Basically, the client should be trying to utilize remaining client bandwidth to speed up upcoming requests and decrease speed spikes as much as possible. An additional feature to achieve this would be to fetch different parts of the same chunk from different chunkservers when it is preferrable to prefetching like on the very first chunk of a file.

[FlorianHeigl]

I mostly wish for the multifetching. i.e. with a butch mechanism on the client, one bucket per chunkserver. It can start requesting from 4 servers, and if it sees that one performs at a lower rate, adjust how much it fetches from that one. the rate can be easily tracked continuously, and it would allow to safely fetch parallel etc. (parallel. not just round-robin. really parallel. ;-)