paper: what are the key principles of programmable storage

michaelsevilla / malacology-popper

2 stars 0 forks source link

paper: what are the key principles of programmable storage #79

Closed michaelsevilla closed 8 years ago

michaelsevilla commented 8 years ago

This should be as "ceph independent" as possible.

michaelsevilla commented 8 years ago

Before mentioning how your framework is implemented, you should first mention that you define (blah blah) our principles/concepts of programmability (if possible). So the reader knows you are describing a new set of ideas, then create a framework to implement and demonstrate those ideas in Ceph.

Jeff

ivotron commented 8 years ago

Proposal for refining the definition: "storage system that exposes existing basic internal abstractions and services so that users can specialize them to define new domain-specific functionality".

Should we add something like the following? "A programmable system guarantees these interfaces provide exactly the same functionality as when used internally (i.e. exposing them has no implications on their correctness)". What I'm trying to say is that exposing them doesn't change the code paths that they take.

ivotron commented 8 years ago

There's an issue of how basic are the abstractions that get exposed. In principle, if consensus is correctly exposed, one could implement versioning and capabilities on top of it. So it seems that we have first-, second-, third-order type of abstractions. Should we make the distinction?

ivotron commented 8 years ago

Other things that can be exposed:

data placement (e.g. making CRUSH programmable w.r.t. object placement).
OSD-level state (e.g. an in-memory, per-OSD counter). I think Noah did this for one of his projects.
snapshots (e.g. defining user-defined logic to determine when to snapshot pools).

michaelsevilla commented 8 years ago

[ ] Programmable storage exposes these interfaces to enable. These building blocks are examples (but is not comprehensive). Our experience implementing services shows that these are good interfaces. (Peter)
[ ] Software defined storage definition in the related work: "What people consider software defined storage". This paper defines it be.
[ ] How basic is the component that you are exposing.
[ ] Programmable storage changes what you can do with a system.

"Expose to enable re-use" -- maybe throw in "interface" -- Peter

michaelsevilla commented 8 years ago

Addressed in 575c91015a489369afb161e4d0ae7b633b5f06cd.