JCache CacheManager customization should happen before named caches are created

[jxblum]

Currently, Spring Boot auto-configuration applies customization to the JCache CacheManager after the creation of (any) configured, "named" caches declared with the spring.cache.cache-names property in Spring Boot application.properties.

If the Spring Boot CacheManagerCustomizer(s) for the JCache CacheManager were to wrap ("decorate") the underlying JCache caching provider's CacheManager implementation (for example: Hazelcast) so that instances of JCache Caches created with the custom wrapped/decorated CacheManager using CacheManager.createCache(..) could also be wrapped and decorated as well, and most likely the case, then the current order for customization vs. cache creation in Spring Boot does not currently allow this to happen.

This order of customization is in contrast to the inverse behavior of Spring Boot's auto-configuration that applies customization to Spring's own CacheManager (Adapter) implementation for the underlying JCache CacheManager created/customized in the dependent bean definition as seen in the Spring CacheManager bean definition.

By applying customization to the Spring JCacheCacheManager immediately after creation, this ensures that the Spring CacheManager will be customized before any Spring Caches are created.

I'd also argue that there do exist cases where wrapping the JCache CacheManager implementation (rather than Spring's CacheManager implementation) is advantageous and even necessary, since other libraries or frameworks (e.g. Hibernate) also depend on caching, and do so using the JCache API.

[jxblum]

FTR, I uncovered this limitation when I was trying to "customize" the JCache CacheManager by decorating the JCache CacheManager caching provider implementation (in my case, Hazelcast) to return custom/decorated JCache Cache instances from this decorated JCache CacheManager (wrapper), where I then encountered:

Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.boot.autoconfigure.cache.CacheManagerCustomizers]: 
Factory method 'cacheManagerCustomizers' threw exception with message: Error creating bean with name 'springCacheManagerCustomizerTwo' defined in io.cacheconsistency.prototype.spring.cache.annotation.BootConcurrencyControlledCachingConfiguration: 
Unsatisfied dependency expressed through method 'springCacheManagerCustomizerTwo' parameter 0: 
Error creating bean with name 'userRepository' defined in io.cacheconsistency.prototype.app.repo.UserRepository defined in @EnableJpaRepositories declared on ApplicationConfiguration: Cannot resolve reference to bean 'jpaSharedEM#0' while setting bean property 'entityManager'
...
..
.
Caused by: org.springframework.beans.factory.BeanCreationException: 
Error creating bean with name 'jpaSharedEM#0': 
Cannot resolve reference to bean 'entityManagerFactory' while setting constructor argument
...
..
.
Caused by: org.springframework.beans.factory.BeanCurrentlyInCreationException: 
Error creating bean with name 'cacheManager': 
Requested bean is currently in creation: 
Is there an unresolvable circular reference?
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.beforeSingletonCreation(DefaultSingletonBeanRegistry.java:355)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:227)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:324)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:313)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200)
    at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:365)
    ... 160 more

Which is the same issue that the user had in Issue #15359.

My circular dependency involved:

JCache CacheManager      -> (JCache) CacheManagerCustomizer (here) to create a CustomJCacheCacheManager (wrapping the caching provider's JCache CacheManager)       -> SD JPA Repository       -> JPA / Hibernate (involving L2 cache)       -> JCache CacheManager (back here)

I then took a different approach to properly address this circular dependency when I then realized that the "customization" would not work in any case, given the description of the limitation above.

[jxblum]

NOTE: My cyclic dependency was even nicely demonstrated by Spring Boot...

The dependencies of some of the beans in the application context form a cycle:

┌─────┐
|  cacheManager defined in class path resource [org/springframework/boot/autoconfigure/cache/JCacheCacheConfiguration.class]
↑     ↓
|  cacheManagerCustomizers defined in class path resource [org/springframework/boot/autoconfigure/cache/CacheAutoConfiguration.class]
↑     ↓
|  springCacheManagerCustomizerTwo defined in io.cacheconsistency.prototype.spring.cache.annotation.BootConcurrencyControlledCachingConfiguration
↑     ↓
|  userRepository defined in io.cacheconsistency.prototype.app.repo.UserRepository defined in @EnableJpaRepositories declared on ApplicationConfiguration
↑     ↓
|  jpaSharedEM#0
└─────┘

[philwebb]

@jxblum Have you got a sample project that shows what you're trying to do? I'm not sure I totally understand all of the moving parts.

I don't think that we can change the JCacheManagerCustomizer so it's called before the jCacheCacheManager.createCache calls because it could break back-compatibility. We could perhaps introduce an additional customize method that we could call earlier, but I'd like to understand the use-case a bit more before we consider doing that.

[jxblum]

@philwebb - Happy to discuss my UC more offline (will reach out to you directly). It is a bit complex to explain here as well as confidential ATM.

[snicoll]

There's quite a back and forth, but this PR has some background as why we're doing things this way: https://github.com/spring-projects/spring-boot/pull/28498

Besides back compatibility, I am not sure we should change that. And if we do we might just as well introduce more options to configure cache names from properties, or remove the cache names property altogether, or make it cache provider specific.

[jxblum]

In all fairness, and after thinking about this a bit more, I suppose it is also possible to delay the JCache Cache customization/decoration until the "named" (JCache) Cache instance is actually requested, such as through a (in)direct call to the JCache CacheManager.getCache(name) method, which is called by Spring's JCacheCacheManager (Spring CacheManager Adapter implementation) in the loadCaches(..) method, here.

Of course, I don't recall precisely when Framework (or perhaps Boot in this case) would call loadCaches() during the initialization lifecycle of the container, just that it will.

[jxblum]

@snicoll - My UC, and many UCs for that matter, has less to do with creating caches up front (using the spring.cache.cache-names property) than it does with simply being able to "intercept" and customize/decorate the (eventual) Cache instance that gets created in the first place.

I simply used this property to conveniently create caches for the underlying caching providers in a somewhat generic manner since I am testing my prototype across multiple caching providers.

All I will say is, when it comes to "consistency", then it is crucial that any system/application architectural component that acquires a handle to a Cache instance does so with "the" customized (decorated) Cache instance.

[jxblum]

@snicoll - After a quick glance and brief review of #28498 (a quite lengthy thread of interactions), I fail to see how customizing the JCache CacheManager before any Cache instances get created and #28498 is related to my ask at all?

In fact, I'd argue whether it is even advisable to create Cache instances upfront for all (JCache) caching providers like this in all cases, as you have alluded to.

Not all caching providers are equal, not by far. Some, like Hazelcast, GridGain, GemFire/Geode, etc, depending on the "named" cache, may perform a very expensive Get Initial Image (GII), that could (block and) delay startup of the Spring Boot application (or minimally make the "named" cache unavailable, particularly in premature @Cacheable method call, until GII is complete). This is particularly true with Hazelcast, GemFire/Geode, etc, in the embedded arrangement. This could even manifest itself differently in different environments (DEV vs. QA vs. PROD), and would be easy for developers to miss before deploying their apps to production.

So, I lean towards, as you say, "...or make it cache provider specific", TBH.

NOTE: As an aside, cache2k is not unique in its feature offering either ("refresh ahead, resilience and exception support"). Hazelcast supports all of these, plus more (for example), as do many other IMDGs. Maybe 1 advantage is it relatively small JAR size (400 KB), though.

[jxblum]

FYI, and 1 final thing to note here, I have explored several alternative approaches to my UC/prototype/POC; One that I already have is a working implementation based on direct customization of Spring's Cache instances using a BeanPostProcessor. However, this approach overall, in many situations, is a bit less than ideal.

I explored Spring Boot's extension points because of the ability to "customize" not only Spring Caching infra components (i.e. using CacheManagerCustomizers) but also JSR-107 JCache components (i.e. with JCacheManagerCustomizer), which being a Java standard specification, then it is more likely that other libraries and frameworks (like Hibernate) will integrate with.

Unfortunately, even the JSR-107 JCache SPI provides no such opportunities for extensions/plugins, which makes Spring Boot's customization offering very attractive and welcomed... needless to say, a really nice and useful feature!

I even explored directly adapting the underlying caching provider's actual cache implementation, but in the end having a standard like the JCache API makes the solution more universally portable.

Truthfully, it is a balancing act between finding the right level of abstraction (e.g. Spring's Cache Abstraction vs. the JCache API, both covering a wide array of caching providers) and getting too low-level, such as with Hazelcast and Distributed Map instances and MapInterceptors (oh my), which does not quite work either, hence the reason I chose JCache.

[snicoll]

I think there is a reason why there's isn't such extension point in the JCache API. Reading all of that, I am not quite sure I fully understood what you're trying to do, or why the current arrangement is not valid.

As Phil asked, please share a small sample that reproduces the scenario you've described. It shouldn't be your current project as we'd like to focus on the ask without dealing with the specifics of your project.

[jxblum]

In a nutshell, and to be a bit more concrete, my UC would be equivalent to (which is completely possible with JCache alone, it is just more "manual" work that is inconvenient compared with Spring Boot's auto-configuration arrangement):

Given:

class CustomCachingProvider implements javax.caching.spi.CachingProvider { ... }

Or (simply just, possible with Spring Boot):

class CustomCacheManager implements javax.cache.CacheManager { ... }

Then (here):

CachingProvider cachingProvider = Caching.getCachingProvider();

CustomCachingProvider customCachingProvider = 
    CustomCachingProvider.wrap(cachingProvider, [<other essential dependencies>]);

Or (here):

CachingProvider cachingProvider = Caching.getCachingProvider(cachingProviderFqn);

""    ""

As you can imagine, then anything returned by the custom/wrapped CachingProvider would be a custom/wrapped CacheManager, which in turn would return custom/wrapped Cache instances.

In other words (and for instance):

class CustomCacheManager implements javax.cache.CacheManager {

    // impl similar for getCache(String, Class<K>, Class<V>)
    public CacheManager getCache(String name) {
        return CustomCache.wrap(wrappedCacheManager.getCache(name), 
            [<passing along other essential dependencies>]);
    }

...
}

And, then:

class CustomCache implements javax.cache.Cache {

    public Object get(Object key) {
        // pre-processing
        Object value = wrappedCache.get(key);
        // post-processing (potentially modifying value)
        return value;
    }
}

Fortunately, and unlike other libraries or frameworks (e.g. Hibernate) integrating with JCache, Spring Boot nicely provides a means to at least customize the JCache CacheManager returned from the SPI (i.e. Caching), which is enough.

I just need the JCache CacheManager to be customized (i.e. "wrapped") before any Caches instances get created (again here).

I am not at the liberty to talk specifics, but can share an example (test) if this would help.

[snicoll]

I am not at the liberty to talk specifics, but can share an example (test) if this would help.

That's what we're asking (a small sample we can run ourselves). So, yes please.

[snicoll]

I am back at a keyboard and had a chance to review this a bit more. I don't think we should be exposing a high-level contract to wrap a CacheManager. I am not sure I get how JCacheManagerCustomizer or CacheManagerCustomizer would help you since they don't allow to return a different instance. It looks like you want the javax.cache.CacheManager bean to be exposed in the bean factory so that you can post-process it before it is customized.

Applying such customizations should only happen on the auto-configured javax.cache.CacheManager. If the user provide its own, then they need to be in full control. For that reason, the customization that is happening now is atomic to the creation of the bean and we can't change that.