search

microsoft / service-fabric

Service Fabric is a distributed systems platform for packaging, deploying, and managing stateless and stateful distributed applications and containers at large scale.
https://docs.microsoft.com/en-us/azure/service-fabric/
MIT License
3.03k stars 399 forks source link

Feature request: Streaming blob reliable collection #920

Open mkosieradzki opened 7 years ago

mkosieradzki commented 7 years ago

It would be very useful to have an IReliableStreamDictionary.

Such string-keyed collection should allow inserting, removing, replacing and reading blob streams. In the minimum scenario such collection should be replicated on the key-level and can have entry size limited according to the replication traffic limits. In an optimal scenario it is replicated on the block-level.

Why is it better than IReliableDictionary<string, Blob>? Because it eliminates serialization overhead and the need to read entire blob into memory to do anything with it.

Why is it better than external blob storage e.g. Azure Blob Storage? Because it is automatically consistent with other collections in a specific partition. It is easy to maintain partition-consistent backups and also easily handle consistent geo-replication scenarios using the SF Backup feature.

Other benefits includes: simple coding model and developer experience, potential performance gains, etc.

masnider commented 7 years ago

We don't have plans for anything like this at this time, but it seems like a neat idea.