The New <IPublishedContentCache> NOT is refreshing cache for Subscriber servers on load balanced environment

[IbrahimMNada]

Which Umbraco version are you using? (Please write the exact version, example: 10.1.0)

15+

Bug summary

When having multiple nodes , one is SchedulingPublisher and the others are Subscriber ,using the extension method SetServerRegistrar .

and when you change a nodes content at the Backoffice, there are no changes at the Subscriber nodes we need to reset the cache for it to be reflected.

My config like follows :

        "DatabaseServerMessenger": {
          "MaxProcessingInstructionCount": 1000,
          "TimeBetweenPruneOperations": "00:01:00",
          "TimeBetweenSyncOperations": "00:00:01",
          "TimeToRetainInstructions": "1.00:00:00"
        }

Specifics

No response

Steps to reproduce

Having 2 nodes , one is SchedulingPublisher and another is Subscriber.

change content at the SchedulingPublisher ,

nothing is changed on the Subscriber

Expected result / actual result

Expected result : changes should be reflected.

actual result: No changes

---

This item has been added to our backlog AB#45026

[github-actions[bot]]

Hi there @IbrahimMNada!

Firstly, a big thank you for raising this issue. Every piece of feedback we receive helps us to make Umbraco better.

We really appreciate your patience while we wait for our team to have a look at this but we wanted to let you know that we see this and share with you the plan for what comes next.

• We'll assess whether this issue relates to something that has already been fixed in a later version of the release that it has been raised for.
• If it's a bug, is it related to a release that we are actively supporting or is it related to a release that's in the end-of-life or security-only phase?
• We'll replicate the issue to ensure that the problem is as described.
• We'll decide whether the behavior is an issue or if the behavior is intended.

We wish we could work with everyone directly and assess your issue immediately but we're in the fortunate position of having lots of contributions to work with and only a few humans who are able to do it. We are making progress though and in the meantime, we will keep you in the loop and let you know when we have any questions.

Thanks, from your friendly Umbraco GitHub bot :robot: :slightly_smiling_face:

[IbrahimMNada]

After some investigations it turned out that the HybridCache when its in memory , there is no way for the subscribers to know that it changed , since in 15- the database was used as a single source of integrity.

We need to define a way to notify the subscribers that cache of the publisher has been changed in order to delete the current in memory cached keys I searched .dotnet core for any hints regarding the hybrid cache behind a load balancer and i didn't found much info.

this is a pretty interesting topic I would gladly help you

Hello ,@Zeegaan , this maybe interesting to you since you're who developed this in v15

I can think of a solution for this and my thoughts goes like follows:

1. we need to use webhooks call back endpoints between the publisher and subscriber.
2. at the configuration level which is the cache configuration introduced in this PR https://github.com/umbraco/Umbraco-CMS/pull/16938
3. Add there a new filed called Role it defaulted to single
4. A. If its Publisher then we need to add also a list of URLS in order to ping them once ContentCacheRefresherNotification is fired. via a web api that could only be registered for the subscribers
5. B. If its subscriber we register a new endpoint that used to be pinged when ContentCacheRefresherNotification at the publisher and each node will reset cache by in order ro clear the in memory state. and when its out-of-process caching whe publisher will reset our own cache provider weather its redis ,sql stc

Regards

[Zeegaan]

Heyo 👋 Great to hear your thoughts on this 😁 We've already been looking into a solution 🕵️ This will include using CacheRefresher so it aligns with the old behavior 👍 When cache tags gets implemented for the HybridCache it will get a lot easier to evict stuff luckily 😁

[IbrahimMNada]

I worked around it with something similar to what you have said , this is my work for the time being ,

I connected it with a web-hook , if you find it worth doing i will be more than happy to help with it

using Microsoft.Extensions.Caching.Hybrid;
using Umbraco.Cms.Core.Cache;
using Umbraco.Cms.Core.Events;
using Umbraco.Cms.Core.Notifications;
using Umbraco.Cms.Core.Services.Changes;
using Umbraco.Cms.Core.Services;
using Umbraco.Cms.Core.Sync;

namespace Tanmiah.External.Cms.Web.DeliveryApi.Composers
{
    public class ContentCacheRefresherNotificationHandler : INotificationAsyncHandler<ContentCacheRefresherNotification>
    {
        private readonly HybridCache _hybridCache;
        private readonly IServerRoleAccessor _serverRoleAccessor;
        public ContentCacheRefresherNotificationHandler(HybridCache hybridCache, IServerRoleAccessor serverRoleAccessor)
        {
            _hybridCache = hybridCache;
            _serverRoleAccessor = serverRoleAccessor;
        }
        public async Task HandleAsync(ContentCacheRefresherNotification notification, CancellationToken cancellationToken)
        {
            if (_serverRoleAccessor.CurrentServerRole == ServerRole.SchedulingPublisher)
                return;

            if (notification.MessageObject is not ContentCacheRefresher.JsonPayload[] payloads)
            {
                return;
            }
            foreach (ContentCacheRefresher.JsonPayload payload in payloads)
            {
                if (payload.ChangeTypes is not TreeChangeTypes.RefreshNode or TreeChangeTypes.RefreshBranch)
                {
                    return;
                }
                await _hybridCache.RemoveAsync(payload.Key.Value.ToString() + "+draft");
                await _hybridCache.RemoveAsync(payload.Key.Value.ToString());
            }
        }
    }
}

[IbrahimMNada]

Even with tags, I don't think it will be very helpful when it comes to in memory caching in hybrid caching.

How will the first node get access to the other node ? we need some sort of intercommunication between nodes in the context of hybrid caching

I have open this proposal on Dotnet extensions repo https://github.com/dotnet/extensions/issues/5517 hope it will start shaking things up ,