search

RaasAhsan / sion

Distributed file system for many large files
1 stars 0 forks source link

Chunk freezing #22

Open RaasAhsan opened 2 years ago

RaasAhsan commented 2 years ago

A chunk can be frozen by a client meaning it can no longer be appended to. I think we don't need to enforce the full size limit and accept some internal fragmentation instead, as this means the contents of short appends remain contiguous within a single chunk.

RaasAhsan commented 2 years ago

Freezing a chunk should return the chunk ID of the next allocated chunk

RaasAhsan commented 2 years ago

When a chunk is frozen, a storage node may still continue to report updates to size?

RaasAhsan commented 2 years ago
  1. When a file is created, the first chunk is automatically allocated and returned.
  2. To add a new chunk to the tail of the file, the current tail must be frozen. By passing this information up, we can atomically check the current tail (with a CAS probably) and create a new chunk.
    • It doesn't really make sense to append a new chunk on a file without first knowing if the parent chunk is full
RaasAhsan commented 2 years ago

How does chunk padding interact with file size?