feature: Add blob and blobfs support

[Xuanwo]

Is your feature request related to a problem? Please describe.

I want to build something around SDPK blobstore. But spdk-rs doesn't support neither blob or blobfs API.

Describe the solution you'd like

I plan to expose blob and blobfs APIs in spdk-rs

Describe alternatives you've considered

Nope.

Additional context

Nope.

[tiagolobocastro]

@Xuanwo just to confirm you mean to say you'll be raising a PR to add this functionality?

[Xuanwo]

@Xuanwo just to confirm you mean to say you'll be raising a PR to add this functionality?

Yes. I'm willing to implement this feature if it looks good to you.

[orville-wright]

This feels like something that would be very useful. Especially BlobFS.

[tiagolobocastro]

@Xuanwo just to confirm you mean to say you'll be raising a PR to add this functionality?

Yes. I'm willing to implement this feature if it looks good to you.

Sounds good @Xuanwo

[orville-wright]

Hey @Xuanwo - There are a couple of SPDK related projects that utilize BlockFS and have an API for integration. ROcksDB , Fuse3 and EVFS are 3 that I know of....

RocksBD has only integrated the db_bench benchmarking tool with BlobFS. RocksDB may be a good example to review as they have clearly done some good SPDK BlobFS work.

• I do like the idea of layering a smart very lightweight embedded Persistent KV storage database with BlobFS. This feels like a very powerful combination solution that would be be very useful.
• OpenEBS would very much like to utilize this kind of tech stack in our future roadmap plans for a Native Object Storage service. Where the Object Store must have a Metadata database to manage the critical metadata elements of the underlying object storage data records. This metadata information is stored separately from the actual object blob data, allowing the system to quickly and efficiently locate and retrieve objects based on their metadata attributes. You also represent & manage the Object Storage filesystem structure inside the database becasue object storage can't do that natively (its not a filesystem). This filesystem is under your full control and doesn't need to be POSIX. It just has to provide the object stores metadata entities in a standard filesystem tree structure. RocksDB + BlobFS might be fantastic for this & a lot of projects could use this.
  • RocksDB is a very modern database...
  • It is very high performance
  • Optimized for NVMe and SSD Flash
  • Very adaptable for differ I/O workloads and use-cases
  • Its based on LevelDB but advances the design... 10X faster writes speed, 7x lower Write Amplification, Less stalls, Bloom Filter scans,
  • info here: https://rocksdb.org/docs/getting-started.html

FUSE3 also has some support for BlobFS, but its seems to not be very well documented.

• The Fuse3 SPDK capability provides the ability to mount a BlobFS on a given device to a host path by FUSE.
  • Then a new thread is created and is dedicated for one mountpoint to handle FUSE requests by the blobfs API.
  • I personally do not like using FUSE inside a storage product, as the overhead and performance impacts are bad.
  • I like the idea of a lightweight filesystem running on BlobFS as that would be a great feature that could be used for many use-cases. But... I don't like the idea of a huge fully featured POSIX compliant filesystem (like ext4) running inside SPDK as that feels like overkill and a lit of headache to deal with. Maybe one of the new modern Flash filesystems would be good (e.g. SSDFS or F2FS would be better. I don't know).

IBM Research in Tokyo did a great SPDK based project called: EvFS: User-level, Event-Driven File System for Non-Volatile Memory. I cant find the code, so I don't think its public.

• The EvFS team build a dynamic link library
• The library exposes POSIX
• They designed and built APIs to eliminate system calls
• Its built on top of SPDK for NVMe
• Whitepaper is here: https://www.usenix.org/system/files/hotstorage19-paper-yoshimura.pdf

[Xuanwo]

Thanks @orville-wright for providing the nice summary!

I'm from the Apache OpenDAL community, and we're exploring this field to see if we can create something cool. I believe we can provide support for blobfs too.

[orville-wright]

Great to know. Thx and it's awesome to meet you ! 😄

I wasn't aware of OpenDAL. Wow, that is a very big and bold project with a lot of complex data/storage layers all integrated together with some very modern data-centric language bindings. - Nice.

It's obvious that you guys at OpenDAL would definitely have a need / use for BlobFS.

• I feel a bit lame now, doing that big write-up for you. sorry about that... you are clearly in the industry like us here at OpenEBS. - We truly get Data and we are the pioneering inventing team behind Stateful Persistent Container Native Storage for K8s.

I am 'Head of Product Mgmt & Strategy' at DataCore and for OpenEBS. DataCore is a key company sponsoring OpenEBS and it provides a lot of eng/dev resources for the project & community. I have a long history designing, developing & bringing to market Storage and Data platforms.

I think there is a good intersection between OpenDAL and OpenEBS. - Your BlobFS API work will definitely be useful in both projects. I support that and can help with your efforts in our community and project.

• Just let me know what you need.