BUG: Major performance decreasement - Flux caching and PreviewRendering

[derhansen]

I noticed a major performance decreasement in TYPO3 backend after updating a flux website from TYPO3 v11 to v12. Profiling with XDebug resulted in, that the flux integration most likely is responsible for the performance drop. To me it seems, that the caching in flux is not working like in flux v9, since neither cache tables nor cache files from flux exist in v10.

The resolveSortingValues function in SpooledConfigurationApplicator for example consumes parts of the execution time. The function tries to cache its result, which however fails, as it uses Typo3DatabaseBackend instead of the default SimpleFileBackend configured in ext_localconf.php. The method Typo3DatabaseBackend->remove() here fails with an exception (since the table does not exist). This problem has also been reported in #2125 and fixed with a try/catch in https://github.com/FluidTYPO3/flux/commit/843526a2e4722d002254ea6e8be367d04e1e0e27

The try/catch does however only suppress the problem and caching does not work in ext:flux 10.x most likely since the extraction of the cache functionality to CacheService (https://github.com/FluidTYPO3/flux/commit/74ff8ca1883225088d46624ec9cb68ef024bc19d).

Reason for the problem may be, that the cache is registered/used before ext_localconf.php or TCA is loaded and TYPO3 falls back to its default cache configuration (VariableFrontend and Typo3DatabaseBackend).

However, manually creating cache_flux table do not resolve the performance problem. It also seems, that CType form data are also not cached any more (maybe introduced with https://github.com/FluidTYPO3/flux/commit/c19925d32a0113fc9ccd5eeab6516031a5fd7e2c).

[derhansen]

In addition: It also seems, that the preview rendering in PageContentPreviewRenderingEventListener is consuming a lot of the execution time. Simply returning early in renderPreview will result in execution times comparable to flux v9

[NamelessCoder]

I'm unable to reproduce a major problem with performance in the caching layer. I've ruled out the possibility that any unexpected fallback to DB backend happens - the cache frontends are injected with DI and would throw a hard error if either of the caches (transient or persistent) are not configured. The only cause I can think of which would make these caches use a different backend is if the cache backend was indeed configured with a different backend.

Re: the remove function: it would only be called in two circumstances:

• Once immediately after registering all provider extensions, explicitly to remove the cached view paths, because those may be different later through TypoScript overrides. This would not result in any (major) performance drawbacks (the only extra CPU cost in incurs is bound to happen anyway; TS configured paths need to be read regardless).
• Once on-demand if you flush the TYPO3 caches or make a chance to a DB-stored Flux definition.

If the remove function is the only one to trigger an exception and neither getFromCaches nor setInCaches does this, then this leads me to suspect that somewhere between the first group of cache fetches/updates and the second single cache removal, the cache backend is somehow changed. I would not expect this to even be possible unless it is during a request that truncates the DI cache (obviously I tested that CacheService does get the right backend type in this particular request - it does, at least on my system). So I have no idea how it could happen that only the remove function throws this exception (even though it is caught and suppressed like the other cache functions).

The caches would actually have to be only partially configured (cache config array set, but backend option missing) or explicitly configured to use a DB backend - otherwise you would receive a "no such cache" exception before the CacheService instance is even created (and the remove function would thus never be called). So the explanation is most likely not that the entire cache configuration is missing / not loaded. It's possible it is explicitly misconfigured to use DB but that would have to be outside of Flux (and shouldn't be done: SimpleFileBackend is much more efficient for this purpose).

So I'm currently drawing a blank as to what's going on with your setup. I'd need you on your end to determine how exactly CacheService ends up configured with a flux cache that uses an unexpected cache backend. The cache backend would have to be wrong already when the CacheService constructor is called...

All that being said, here is what I did find:

• One possible difference from Flux 9 to 10 is that extracted Form instances are no longer retained in a transient cache if the Form is configured as static: false. This could potentially cause the same content template to be executed multiple times. Technically this isn't necessary and Forms can be allowed to be cached in transient (runtime) caches even if they do not specify that they are static: true. Allowing them to be transient cached should result in a ~25% or better improvement for backend requests (higher the more templates and content instances you have).
• Unless your flux:form implementations use dynamically configured fields (conditions in Fluid template based on variables that may result in a different Form composition, for example in different page tree sections) you should ensure that the flux:form tag specifies options="{static: 1}". This informs Flux that once the Form instance is extracted from the template, a serialized representation of the Form may be stored in persistent caches which prevents the relatively heavy Fluid rendering and extraction steps when Flux tries to get the Form from a template.

[derhansen]

Thanks for your reply. First of all, we can remove the term major here, since I found out, that TYPO3_CONTEXT Development may have been a reason for the much slower response times. After changing this to TYPO3_CONTEXT Production, the backend was kind of faster, but not as fast as in TYPO3 v11 with flux v9.

I think, the last part of you answer could explain, why the backend although is slower than before. The website uses ~60 content elements based on flux and some pages heavily rely on nested content structures based on flux (e.g. multi-column elements, with additional flex elements inside) with about 50-100 content elements on some pages. I tried to use options="{static: 1}" to get the Forms cached as in v9, but with no success. Here I again end up in the try/catch part of the exception from the CacheService.

Cache configuration as shown in Configuration module:

Persistent Cache in constructor of CacheService

So here the Typo3DatabaseBackend is used, although SimpleFileBackend is configured. Since the Typo3DatabaseBackend is not configured (no tables available), nothing is cached in persistent cache and the exception is thrown (and catched).

Is the $persistentCacheinjected for you as SimpleFileBackend, when you set a breakpoint in the constructor of CacheService?

[NamelessCoder]

Is the $persistentCacheinjected for you as SimpleFileBackend, when you set a breakpoint in the constructor of CacheService?

Yes, in every context that I tested, it is the correct type.

When you encounter the DB backend in your breakpoint, did you try dumping TYPO3_CONF_VARS as well to study the cache configuration? Is it incomplete or using an unexpected backend? What you see in the "Configuration" backend module may not be the whole truth since this configuration array may be mutated and part of it may be conditional (through use of third-party code or configuration; the core doesn't mutate it, nor does Flux).

Did you also try adding breakpoints in for example the DB cache backend constructor and check the execution stack to find out how/when it is constructed to be used as backend for the flux cache?

Is there anything - hook, third party extension, ENV-dependent configuration or other - that would result in cache configurations being changed? For example, extensions that may change the cache backends in bulk, apply different configurations for them, override CacheManager, or similar?

There has to be some aspect that's changing how your setup assigns cache backends because this wouldn't be caused by circumstantial failure to read the cache configurations (as mentioned, that would produce a much earlier and uncaught exception). Something that exists in your setup but does not exist in a vanilla setup that only has Flux installed.

Obviously, the static option has no practical effect if the cache backend is broken. But fix that issue and static would probably increase your performance a lot.

[NamelessCoder]

I tried to use options="{static: 1}" to get the Forms cached as in v9

Just for the record: Flux v9 did not persistently cache forms unless you set static: true, but in v10 it doesn't transiently cache them unless static is true. So the default persistent cache behavior hasn't changed.

[derhansen]

Thanks to your help, I found the problem now.

While debugging TYPO3_CONF_VARS in the constructor of CacheService I noticed, that the flux cache entry was missing. After defining a dependency to "fluidtypo3/flux": "^10.0.10", in the composer.json of the sitepackages, the cache was initialized correctly and the backend performance was good.