Use of CatalogDataResourceProviderManager and how to combine with product nodes

[geertsmichael]

Expected Behaviour

In our set-up we have the CIF connector installed and use Product bindings in /var/commerce/products to virtually connect to Magento. No nodes will ever be created for the Magento use case under /var/commerce/products/our-magento-binding/.

But we also have a brand that wants to use the traditional way of handling products by importing products under /var/commerce/products/traditional-imported-products/.

Actual Behaviour

When updating, creating or removing a node in /var/commerce/products/traditional-imported-products/, we see the following stack trace in the log files and the system becomes very slow due to huge amount of data.

07.05.2021 15:04:42.733 WARN [sling-oak-observation-10] org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex Index-Traversed 20000 nodes with filter Filter(query=SELECT FROM [sling:Folder] WHERE ISDESCENDANTNODE('/var/commerce/products') AND [sling:Folder].'cq:catalogDataResourceProviderFactory' IS NOT NULL, path=/var/commerce/products//, property=[cq:catalogDataResourceProviderFactory=[is not null]])

We also are wondering what this class is exactly doing: https://github.com/adobe/commerce-cif-connector/blob/7d56aa839ef646edb8d47e4fe1ead91d97349182/bundles/cif-virtual-catalog/src/main/java/com/adobe/cq/commerce/virtual/catalog/data/impl/CatalogDataResourceProviderManagerImpl.java (this is the class causing the log file entries) Why is this needed for the Magento integration and shouldn't it be configured differently to only be triggered for the Magento bindings only?

Steps to Reproduce

Create a traditional folder under /var/commerce/products and create product nodes.

Platform and Version

CIF 0.9.0, AEM 6.4.5

Sample Code that illustrates the problem

Logs taken while reproducing problem

07.05.2021 15:04:42.733 WARN [sling-oak-observation-10] org.apache.jackrabbit.oak.plugins.index.lucene.LucenePropertyIndex Index-Traversed 20000 nodes with filter Filter(query=SELECT FROM [sling:Folder] WHERE ISDESCENDANTNODE('/var/commerce/products') AND [sling:Folder].'cq:catalogDataResourceProviderFactory' IS NOT NULL, path=/var/commerce/products//, property=[cq:catalogDataResourceProviderFactory=[is not null]])

[mhaack]

I had a chat with @geertsmichael the issue here is every change in /var/commerce/products triggers the CatalogDataResourceProviderManagerImpl event listener. onEvent method should better filter the events here and not execute if other nodes (other then catalog binding configs) are created/changed/deleted.

[mhaack]

Created CIF-2082 for tracking

[mhaack]

@geertsmichael can you please do some extra testing one which events these issues happens? We currently assume the issue only occurs on changed nodes during the import, but not if new product nodes are added or existing product nodes are removed.