Only InMemoryObjectStorage implements IObjectStorage, no others storage(e.g. file, redis) implementation

[a422241894]

Hi guys, I have found that there is only InMemoryObjectStorage implements IObjectStorage, if our client server crashed, all events in the queue will be lost, there is nowhere to serialize the log events to disk on client-side. If I don't use ExceptionlessClient.Default.Configuration.UseInMemoryStorage(), it still use InMemoryObjectStorage to store client events to the in-memory queue，can anyone help to explain it?

[niemyjski]

Hello,

We have this detailed here: https://github.com/exceptionless/Exceptionless.Net/wiki/Configuration#offline-storage you take a performance hit on double serialization to store it to disk and we feel like it's just not worth it for most cases. Events are submitted on process exit (hopefully, we wire up to all events but yes there can be some hard crashes), and we submit every 10 seconds anything in the queue.

[a422241894]

@niemyjski Thanks for the detailed information, now I have two questions:

1) The queue length in memory is default set to 1000 when I use InMemory storage , if we post 1000+ events to the queue in a few seconds, the most earlier events will be dequeue and deleted, and we will lost these events, are you doing this for performance reasons?

2) My company want to use Exceptionless by self-hosting, we don't want post events by web api, is it possible to post these events to redis server directly in Exceptionless client, and we consume the events in Exceptionless server?

[niemyjski]

1. If for some reason we hit that limit, we discard the oldest one (https://github.com/exceptionless/Exceptionless.Net/blob/e82b5b06b7e709c3c60f7609147978f24f15eb44/src/Exceptionless/Storage/InMemoryObjectStorage.cs#L57). We have deduplication by default and this is not very likely to happen. I've never seen it happen logging throughout our infrastructure. But it could happen.
2. You'd need to write some code to make this happen. You'd need to write your own ISubmissionClient to store these in Redis and then have a job server side that looks for these and enqueues them for processing. It's potentially putting a huge load on redis because the payloads could be quite large. Any reason for not wanting to use web api?

[ejsmith]

2. Is not a good idea. There are all kinds of things we are doing on the server side to process the events. Skipping all that kind of defeats the purpose of using Exceptionless. Did you see that we have a FolderObjectStorage that you can use if you really want to make sure that you don't lose events? Not really sure what you are trying to do overall though.

[niemyjski]

I'm going to close this for the reasons above, we are open to any other feedback or implementation suggestions.