search

project-machine / puzzlefs

A next-generation container filesystem
Apache License 2.0
393 stars 17 forks source link

clarifications about metadata #5

Closed hallyn closed 3 years ago

hallyn commented 3 years ago

I have a few questions about the metadata.md. I'm grouping them here as I'm not sure how related they are:

  1. The comment says that the list of metadatas can be included at offsets or in a blob. "The list of metadatas". But each metadata_ref has its own [local|blob] type. Can some metadatarefs be local and some in blobs, or is the whole list of metadatas to be grouped?
  2. the DIR_LIST_LOOK_BELOW is confusing to me. What does 'below' refer to? The comment says "when set, also look at layers below for the dir list." First off, I assume this is a flag for dir_list.flags? I would assume anything 'below' some dir_list would be nested directories, i.e. /a/b/c under /a/b. But I don't think that's what you mean, since that would simply be implied by the inode types in the dir_list.
tych0 commented 3 years ago

The comment says that the list of metadatas can be included at offsets or in a blob. "The list of metadatas". But each metadata_ref has its own [local|blob] type. Can some metadatarefs be local and some in blobs, or is the whole list of metadatas to be grouped?

The idea is that you can reference something else in the current blob, so we don't need container formats for everything. You can say "this is in the same blob at offset X" and then just seek() there and read whatever it is, or you can say "this is in the blob with digest Y at offset Z", open that one, and seek there. This way we can point into existing metadata blobs for lower layers without having to copy anything.

the DIR_LIST_LOOK_BELOW is confusing to me. What does 'below' refer to? The comment says "when set, also look at layers below for the dir list." First off, I assume this is a flag for dir_list.flags? I would assume anything 'below' some dir_list would be nested directories, i.e. /a/b/c under /a/b. But I don't think that's what you mean, since that would simply be implied by the inode types in the dir_list.

Right, this is for the fs layers below the current one. Motivation here being essentially the same as the above: if only one directory entry in a large directory has changed, we can just say "here's the new inode of X, look at previous layers for everything else".

hallyn commented 3 years ago

essentially the same as the above: if only one directory entry in a large directory has changed, we can just say "here's the new inode of X, look at previous layers for everything else".

Oh - right - I see. Thanks.