MichaCo / CacheManager

CacheManager is an open source caching abstraction layer for .NET written in C#. It supports various cache providers and implements many advanced features.
http://cachemanager.michaco.net
Apache License 2.0
2.34k stars 457 forks source link

Query : Why manually publishing message for Add/Update key and not using keyspace notifications? #244

Closed shah-keyur closed 6 years ago

shah-keyur commented 6 years ago

Hi MichaCo,

Question: Just wondering that why not use the Set ('s') for key space event and manually publishing the change event(NotifyChange) on each add/update/change operation in RedisCacheHandle.cs - SubscribeKeyspaceNotifications()?

Guess: I am just guessing that the event Redis generate does not have much info like Action = add/update ,?

Question: is that on each add/update for all other handler and in case of ELB multiple instances we are removing or evicting the key-value pair. is there reason that why we can not update with the value instead of removing?

Guess: I think the event what Redis published might not have updated value and enough information.

Backplane : I think backplane is only for ELB scenario. Without ELB we don't require the Backplane, right?

The question with code : BaseCacheManager.cs - RegisterCacheBackplane()

TriggerOnAdd,TriggerOnPut,TriggerOnUpdate

What's the use of it and how it will be useful? backplane.Changed += (sender, args) => { EvictFromHandles(args.Key, args.Region, handles(false));

                //TODO: Confirm what it does and then work on it or remove it
                switch (args.Action)
                {
                    case CacheItemChangedEventAction.Add:
                        TriggerOnAdd(args.Key, args.Region, CacheActionEventArgOrigin.Remote);
                        break;

                    case CacheItemChangedEventAction.Put:
                        TriggerOnPut(args.Key, args.Region, CacheActionEventArgOrigin.Remote);
                        break;

                    case CacheItemChangedEventAction.Update:
                        TriggerOnUpdate(args.Key, args.Region, CacheActionEventArgOrigin.Remote);
                        break;
                }
            };

Question : When using the only Redis, the event part gets executed and eats a lot of time to serialize and deserialize? Any way to make it faster? I know the reason that it has to make a check for all the handles and publishing a message.

Appreciate your help and would like to say, you did a fantastic job.

Thanks, Keyur

MichaCo commented 6 years ago

Re 1. Question, why not using keyspace notifications: a) notifications must be enabled on each redis server instance explicitly (and isn't enabled in many cloud solutions). Pub/Sub is usually enabled/available. b) need more / custom information

Re 2. Question (not sure what you mean exactly) It is safer to remove keys in other instance's local in-memory cache instead of getting the value or even transporting the value. the key could already got deleted/updated by another instance while transporting the value. Only the source system (e.g. Redis) should handle the actual value

Re Question 3 (ELB only) Backplane can be useful if you have more than one instance of your app running and have to keep the local in-memory cache kinda synced

Re Question 4 Idk if that's useful for your application logic, those things are used internally. Methods are not internal so that they can be used in case you find them useful 😸

Re Question 5. If you only use Redis (without in-memory cache) You don't need a backplane.

There is some overhead to de/serialize backplane messages. But it is very optimized and is MUCH faster in most cases than the actual Redis operations. If you look at the benchmarks, it can serialize 5-10million messages per second.

Only real cost is network, because we have to send the messages to Redis and then to all subscribed clients. That's why the backplane sends messages in batches (if there are a lot of messages in a short time frame).

In general, not sure if Github issues is the best platform for many questions. I guess SO would have been smarter. This information here gets lost pretty quickly ;)

shah-keyur commented 6 years ago

Hi MichaCo,

All the answers are spot on. Cleared with doubts.

Thank a lot. -Keyur

shah-keyur commented 6 years ago

Hi MichaCo,

A just weird thing I observed is when I test the CacheManger using console app and Web application.

The time taken by Redis is higher than the time taken by Redis in a Web application.

In case you might have observed something similar?

Thanks, Keyur

MichaCo commented 6 years ago

nope no idea