Redirects in Redirect Manager not working until ACS Commons Bundle is Restarted

jaylaw81 commented 1 month ago

Required Information

[x] AEM Version, including Service Packs, Cumulative Fix Packs, etc: 6.5.20.0
[x] ACS AEM Commons Version: 6.3.6
[x] Reproducible on Latest? yes

Expected Behavior

When new redirects are added into author environment and redirects are published. Expecting that redirects are active in Publish environment and then also will get picked up in Dispatcher

Actual Behavior

Redirects that are published are not working on Publisher instance or Dispatcher instance.

Steps to Reproduce

Redirects are added in Author
Redirects get published
Redirects are confirmed to make it to publish environment via crx/de
Redirects however do not work as intended and require the ACS commons bundle to be restarted for Redirects to work

YegorKozlov commented 1 month ago

Is it a sporadic issue or does it happen consistently every time you publishing redirects? I can't reproduce it on my local 6.5.20. Redirect Manager exposes its cache via a JMX bean

Use getRedirectRules(path) to see what is actually loaded:

This method returns a tabular view of what is loaded and being served .

Then clear the cache using invalidateAll() and see if it makes any difference

jaylaw81 commented 1 month ago

We are seeing this occur consistently. We are also in an environment that uses 4 publishers and 4 dispatchers within a load balancer

YegorKozlov commented 1 month ago

I can't reproduce it on my local 6.5.20. Our production runs 6.5.18 and redirects are invalidated just fine.

Could you create a trace logger for the com.adobe.acs.commons.redirects.filter namespace ? I'd like to see what's happening on your publish instances.

Also, do you have a custom configuration for org.apache.felix.eventadmin.impl.EventAdmin ? Redirect Manager listens for replication / change events and I wonder if EventAdmin is configured to ignore them.

YegorKozlov commented 1 month ago

I think something is wrong and it might have to do to SP 6.5.20.

I couldn't reproduce it at first, and today some redirects wouldn't invalidate. Checking the server logs revealed an exception:

04.06.2024 08:26:09.172 *WARN* [sling-oak-observation-18] com.adobe.cq.dam.cfm.graphql.ModelManagerImpl Could not get conf path for model /conf/xxxxx/settings/redirects/redirect-rule-16
java.lang.IllegalArgumentException: Not a valid model path: /conf/xxxxx/settings/redirects/redirect-rule-16
    at com.adobe.cq.dam.cfm.graphql.Util.getConfPathForContentFragmentModel(Util.java:143) [cq-dam-cfm-graphql:0.0.1.CQ650-B0030]
    at com.adobe.cq.dam.cfm.graphql.ModelManagerImpl.invalidate(ModelManagerImpl.java:220) [cq-dam-cfm-graphql:0.0.1.CQ650-B0030]
    at com.adobe.cq.dam.cfm.graphql.ModelChangeListener.handleChange(ModelChangeListener.java:105) [cq-dam-cfm-graphql:0.0.1.CQ650-B0030]
    at com.adobe.cq.dam.cfm.graphql.ModelChangeListener.onChange(ModelChangeListener.java:85) [cq-dam-cfm-graphql:0.0.1.CQ650-B0030]
    at org.apache.sling.resourceresolver.impl.observation.BasicObservationReporter.reportChanges(BasicObservationReporter.java:211) [org.apache.sling.resourceresolver:1.7.10.B006]
    at org.apache.sling.jcr.resource.internal.JcrResourceListener.onEvent(JcrResourceListener.java:155) [org.apache.sling.jcr.resource:3.0.18]
    at org.apache.jackrabbit.commons.observation.ListenerTracker$1.onEvent(ListenerTracker.java:190) [org.apache.jackrabbit.jackrabbit-jcr-commons:2.20.11]
    at org.apache.jackrabbit.oak.jcr.observation.ChangeProcessor.contentChanged(ChangeProcessor.java:513) [org.apache.jackrabbit.oak-jcr:1.22.18]
    at org.apache.jackrabbit.oak.plugins.observation.FilteringDispatcher.contentChanged(FilteringDispatcher.java:52) [org.apache.jackrabbit.oak-core:1.22.18]
    at org.apache.jackrabbit.oak.spi.commit.BackgroundObserver$1$1.call(BackgroundObserver.java:127) [org.apache.jackrabbit.oak-store-spi:1.22.18]
    at org.apache.jackrabbit.oak.spi.commit.BackgroundObserver$1$1.call(BackgroundObserver.java:121) [org.apache.jackrabbit.oak-store-spi:1.22.18]
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)

It's not an ACS-Commons error, it's coming from elsewhere.

It looks like a class from the cq-dam-cfm-graphql bundle listens to changes in /conf/* nodes and fails when the change came from a redirect node. Going by the cq-dam-cfm-graphql name, ModelManagerImpl is supposed to invalidate graphql models only and somehow it catches wrong events.

jaylaw81 commented 1 month ago

I'm not that familiar with those issues. Is this something that needs to be opened with Adobe?

YegorKozlov commented 1 month ago

I'm not that familiar with those issues. Is this something that needs to be opened with Adobe?

yes, it could be a bug in the cq-dam-cfm-graphql bundle. I'd like to do more research when time permits and see if it works in AEM Cloud and if there is a workaround to make it work in 6.5.20.

Adobe-Consulting-Services / acs-aem-commons