search

exceptionless / Exceptionless

Exceptionless application
https://exceptionless.com
Apache License 2.0
2.4k stars 517 forks source link

Investigate using System.Text.Json for storage persistence #831

Open benaadams opened 3 years ago

benaadams commented 3 years ago

It allows for direct async saving to Stream; have added PR to https://github.com/FoundatioFx/Foundatio/pull/259 to support the async mode

Would hopefully reduce the steps in serializing in memory, re-reading and then writing out again image

ejsmith commented 3 years ago

I think it would be hard for us to have the events pipeline be completely stream based, but I wonder if we could have the reader be stream based and then have the pipeline iterate through one event at a time using an IAsyncEnumerable<PersistentEvent> where we just load a single event into memory at a time and process it and then release that memory back into a pool to load the next one.