resource-disaggregation / jiffy

Virtual Memory Abstraction for Serverless Architectures
Apache License 2.0
45 stars 13 forks source link

File parallel read/write #93

Closed charles-typ closed 4 years ago

charles-typ commented 4 years ago

Changes provided in this PR:

Given these two, the partition would act as a transfer station between the client and auto-scaling server. After the auto-scaling server finish adding the blocks, it would notify the partition the new blocks are ready and the partition would notify the client to run the following writes in parallel.

As for reads, we suppose that the file could not read exceed the file size(determined by the write with the max offset), so it would be safe to read in parallel directly since the file has sufficient blocks already.

Two new logic has been added to seek. If the client first seek to an offset outside the file size, then it performs a read, the read will fail. On the other hand, if a write is performed, the client would first allocated enough blocks to both reach the new offset and to have enough space to perform that write. This part of code re-uses the code above.

Handle_redirect in file_client and next_target_str in each file partition is no longer needed

anuragkh commented 4 years ago

Is this PR ready to review? The title still says WIP.

charles-typ commented 4 years ago

Yes, just added some final changes

charles-typ commented 4 years ago

Changes:

  1. Decouple write and add blocks
  2. Change read write signatures
  3. Remove useless members in file_block, simplify the serializer and deserializer for file
  4. Simply file partition logic, totally remove next_targetstr related code
  5. Add new file partition function: register_block, happens in file partition's constructor, ask the first partition its capacity and whether it uses auto_scaling. In this way, the users doesn't need to pass these arguments in the client constructor.
anuragkh commented 4 years ago

Looks good mostly. You can merge after making the minor suggested changes.