search

MichaCo / CacheManager

CacheManager is an open source caching abstraction layer for .NET written in C#. It supports various cache providers and implements many advanced features.
http://cachemanager.michaco.net
Apache License 2.0
2.35k stars 456 forks source link

Support for Protocol Buffers DataReader Extensions for .NET #174

Closed jbparker closed 7 years ago

jbparker commented 7 years ago

Currently, if using ProtoBufSerializer for persistent serialization, the only option for serializing System.Data.DataTable and System.Data.DataSet is using BinarySerializer.

For large DataTable & DataSet objects, this does not serialize well and typically throws an OutOfMemoryException for non-trivially sized collections.

I am currently employing a custom serializer that uses Protocol Buffers DataReader Extensions for .NET to serialize DataTable and DataSet (along with gzip support mentioned on #173).

@MichaCo Same on this issue - would this be desirable to have in the mainline source?

MichaCo commented 7 years ago

@jbparker I don't really think I want this library/use-case in CacheManager because a) The use-case is pretty small. b) I believe that serializing a DataTable or similar directly is a bad idea and should be implemented differently (cache only the data you actually need and create custom contracts for those => easy to serialize). c) It would have to be another CacheManager Nuget (as I don't want to have a reference to it in the current protobuf Nuget). And there is quite some work to be done to maintain, test and release that.

That being said, you can always create your own library which does use that library, or, simply serialize your datatable outside of CacheManager and then cache the resulting byte array...

What do you think?

jbparker commented 7 years ago

Agreed! Especially agree with b) - most of these are legacy application utilizations that shouldn't be used go forward but that are, unfortunately, quite prevalent in our application.

Because of this, I'm good leaving this off - just wanted to check. Thanks for your time!