Blitz can't clear caches for urls containing ü,ö,ë etc

[lenvanessen]

We have a strange issue with Blitz at the moment.

Caches or being generated and served by blitz, and our blitz_caches table counts 4600 rows. However, our blitz_elementcaches counts 0. So whenever the RefreshCacheJob tries to locate the cacheIds it thinks there aren't any.

So the RefreshCacheJob can't find the cacheIds for the page, and the page remains unchanged. Any idea why this might be happening?

[bencroker]

Have you set the cacheElements config setting to false? https://github.com/putyourlightson/craft-blitz/blob/3.6.8/src/config.php#L151

What versions of Blitz and Craft are you running?

[lenvanessen]

@bencroker yes... false, that's most likely the culprit. Changed that and now it works.

[bencroker]

Glad to hear it!

[lenvanessen]

Reopening. Though the elements are now in shown in the blitz_elementcaches table, they are still not cleared properly. What i mentoined "worked" was a small workaround we made.

Any idea?

[lenvanessen]

Blitz 3.6.8 and craft 3.5.8

[bencroker]

How exactly are you testing and what is not working?

[lenvanessen]

@bencroker

1. Clear the full cache
2. Load page once to get it into cache, verify by checking the info cached by blitz..
3. Change title, wait for queue to finish processing
4. Go back to page, old html still there

[bencroker]

Here is how I would suggest testing:

1. Clear the cache
2. Verify that the cache folder is empty
3. Visit the page
4. Verify that the cache folder contains the cached page and check the modified timestamp of index.html
5. Change the entry title and save
6. Wait for the refresh cache job to finish, then check the modified timestamp of index.html to see if it was changed

[lenvanessen]

@bencroker We're not working with a folder, cache is stored in a redis db.

[bencroker]

Ok, are there any other things in your setup that deviate from the defaults?

[lenvanessen]

<?php
/**
 * @copyright Copyright (c) PutYourLightsOn
 */

use putyourlightson\blitz\drivers\storage\FileStorage;
use putyourlightson\blitz\drivers\storage\YiiCacheStorage;

/**
 * Blitz config.php
 *
 * This file exists only as a template for the Blitz settings.
 * It does nothing on its own.
 *
 * Don't edit this file, instead copy it to 'craft/config' as 'blitz.php'
 * and make your changes there to override default settings.
 *
 * Once copied to 'craft/config', this file will be multi-environment aware as
 * well, so you can have different settings groups for each environment, just as
 * you do for 'general.php'
 */

return [
    '*' => [
        // With this setting enabled, Blitz will log detailed messages to `storage/logs/blitz.php`.
        'debug' => true,

        // With this setting enabled, Blitz will begin caching pages according to the included/excluded URI patterns. Disable this setting to prevent Blitz from caching any new pages.
        'cachingEnabled' => getenv('ENABLE_CACHE') == 'true',

        // The URI patterns to include in caching. Set `siteId` to a blank string to indicate all sites.
        'includedUriPatterns' => [
            [
                'siteId' => '',
                'uriPattern' => '.*',
            ],
        ],

        // The URI patterns to exclude from caching (overrides any matching patterns to include). Set `siteId` to a blank string to indicate all sites.
        'excludedUriPatterns' => [
           [
               'siteId' => 2,
               'uriPattern' => '/search',
           ],
           [
               'siteId' => 1,
               'uriPattern' => '/de/search',
           ],
        ],

        // The storage type to use.
        'cacheStorageType' => YiiCacheStorage::class,

        // The storage settings.
        'cacheStorageSettings' => [],

        // The storage type classes to add to the plugin’s default storage types.
        //'cacheStorageTypes' => [],

        // The warmer type to use.
        //'cacheWarmerType' => 'putyourlightson\blitz\drivers\warmers\GuzzleWarmer',

        // The warmer settings.
        'cacheWarmerSettings' => ['concurrency' => 4],

        // The warmer type classes to add to the plugin’s default warmer types.
        //'cacheWarmerTypes' => [],

        // Custom site URIs to warm when either a site or the entire cache is warmed.
        //'customSiteUris' => [
        //    [
        //        'siteId' => 1,
        //        'uri' => 'pages/custom',
        //    ],
        //],

        // The purger type to use.
       // 'cachePurgerType' => 'putyourlightson\blitz\drivers\purgers\CloudflarePurger',

        // The purger settings.
        //'cachePurgerSettings' => [],

        // The purger type classes to add to the plugin’s default purger types.
        //'cachePurgerTypes' => [
        //    'putyourlightson\blitzshell\ShellDeployer',
        //],

        // The deployer type to use.
        //'deployerType' => 'putyourlightson\blitz\drivers\deployers\GitDeployer',

        // The deployer settings.
        //'deployerSettings' => [
        //    'gitRepositories' => [
        //        'site-uid-f64d4923-f64d4923-f64d4923' => [
        //            'repositoryPath' => '@root/path/to/repo',
        //            'branch' => 'master',
        //            'remote' => 'origin',
        //        ],
        //    ],
        //    'commitMessage' => 'Blitz auto commit',
        //    'username' => '',
        //    'personalAccessToken' => '',
        //    'name' => '',
        //    'email' => '',
        //    'commandsBefore' => '',
        //    'commandsAfter' => '',
        //],

        // The deployer type classes to add to the plugin’s default deployer types.
        //'deployerTypes' => [
        //    'putyourlightson\blitzshell\ShellDeployer',
        //],

        // Whether the cache should automatically be cleared when elements are updated.
        'clearCacheAutomatically' => true,

        // Whether the cache should automatically be warmed after clearing.
        'warmCacheAutomatically' => getenv('CACHE_AUTO_WARM') == 'true',

        // Whether the cache shwould automatically be refreshed after a global set is updated.
        //'refreshCacheAutomaticallyForGlobals' => true,

        // Whether URLs with query strings should cached and how.
        // 0: Do not cache URLs with query strings
        // 1: Cache URLs with query strings as unique pages
        // 2: Cache URLs with query strings as the same page
        'queryStringCaching' => 1,

        'excludedQueryStringParams' => ['gclid', 'fbclid', 'utm_campaign', 'utm_medium', 'utm_source', '__geom'],

        // An API key that can be used to clear, flush, warm, or refresh expired cache through a URL (min. 16 characters).
        //'apiKey' => '',

        // A path to the `bin` folder that should be forced.
        //'binPath' => '',

        // Whether elements should be cached in the database.
        'cacheElements' => true,

        // Whether element queries should be cached in the database.
        'cacheElementQueries' => false,

        // The amount of time after which the cache should expire (if not 0). See [[ConfigHelper::durationInSeconds()]] for a list of supported value types.
        //'cacheDuration' => 0,

        // Element types that should not be cached (in addition to the defaults).
        //'nonCacheableElementTypes' => [
        //    'putyourlightson\campaign\elements\ContactElement',
        //],

        // Source ID attributes for element types (in addition to the defaults).
        //'sourceIdAttributes' => [
        //    'putyourlightson\campaign\elements\CampaignElement' => 'campaignTypeId',
        //],

        // The integrations to initialise.
        //'integrations' => [
        //    'putyourlightson\blitz\drivers\integrations\FeedMeIntegration',
        //    'putyourlightson\blitz\drivers\integrations\SeomaticIntegration',
        //],

        // The value to send in the cache control header.
        //'cacheControlHeader' => 'public, s-maxage=31536000, max-age=0',

        // Whether an `X-Powered-By: Blitz` header should be sent.
        //'sendPoweredByHeader' => true,

        // Whether the timestamp and served by comments should be appended to the cached output.
        //'outputComments' => true,

        // The priority to give the refresh cache job (the lower number the number, the higher the priority). Set to `null` to inherit the default priority.
        //'refreshCacheJobPriority' => 10,

        // The priority to give driver jobs (the lower number the number, the higher the priority). Set to `null` to inherit the default priority.
        //'driverJobPriority' => 100,

        // The time in seconds to wait for mutex locks to be released.
        //'mutexTimeout' => 1,

        // The paths to executable shell commands.
        //'commands' => [
        //    'git' => '/usr/bin/git',
        //],
    ],
    'dev' => [
        // The storage type to use.
        'cacheStorageType' => FileStorage::class,

        // The storage settings.
        'cacheStorageSettings' => ['folderPath' => '@webroot/cache/blitz'],
    ],

    'staging' => [
        // The storage type to use.
        'cacheStorageType' => FileStorage::class,

        // The storage settings.
        'cacheStorageSettings' => ['folderPath' => '@webroot/cache/blitz'],
    ],

    'production' => [
        'cachePurgerType' => 'putyourlightson\blitz\drivers\purgers\CloudflarePurger',
    ]
];

[lenvanessen]

Btw, cloudflare is not enabled at the moment. We also worked on the clearRelatedCaches if you can remember, but that doesn't interfere with the refreshCache functionality

[bencroker]

I see you're using YiiCacheStorage in production only. Is the issue happening in the dev or staging environments in which FileStorage is being used?

[lenvanessen]

@bencroker no problems on local with the normal file storage, staging also uses the YiiCacheStorage adaptor as an interface for the cache-stores.

[bencroker]

So if the issue is limited to your production environment then it sounds like it might be a devops issue. If you can't solve it then I'd be happy to schedule some time to help troubleshoot this with you via email or chat.

[lenvanessen]

@bencroker just found out it works for some entries, and others not. As fas as I can see, entries with special characters int he url like ü are failing to clear, others work.

[lenvanessen]

@bencroker and only when using the redis store, when using the regular text files they don't give issues.

[lenvanessen]

Just tested a couple of extra posts, all posts with a ä, ü etc in the url have this issue, none of the others.

[bencroker]

That's very possibly the cause of the issue. Perhaps you can look into whether this is something you can solve in your redis store.

[lenvanessen]

@bencroker turns out its not only redis, also file storage adaptor has this issue as well

This is the url döta-2-braucht-die-replay-takeover-funktion-zurueck-dringend

When Blitz creates the cache, the following files are created:

But when deleting, it tries to delete web/cache/blitz/early-games.test/de/dota-2-braucht-die-replay-takeover-funktion-zurueck-dringend/index.html (without omlaut, but the umlaut is actually stored in the path resulting in Blitz being unable to delete the file)

Looks like a bug in Blitz, just can't pinpoint where its going wrong. Can you take a look?

[bencroker]

Am not seeing this but will run some tests.

Reference: https://github.com/putyourlightson/craft-blitz/issues/222

[bencroker]

Fixed for the file storage driver in https://github.com/putyourlightson/craft-blitz/commit/4305794b87655deffa7744b1a4a6a9f72a32ca99, can you test if this solves the issue on your end?

[lenvanessen]

Hi @bencroker, Sorry for the late reply, I tested it and it works like a charm. Is there a fix for the redis adaptor in the works as well?

[bencroker]

That's next on the list ;)

[hiasl]

Hi @bencroker, 1.) we have the same problem with hungarian special chars 2.) is it expected behavior, that two directories are created instead of one? one with the special chars, one with the encoded chars? same as with @lenvanessen's example above 3.) when are you going to release the bugfix mentioned above? thanks, Matthias

[bencroker]

is it expected behavior, that two directories are created instead of one? one with the special chars, one with the encoded chars?

This was introduced as a fix to the issue when using the file storage driver.

when are you going to release the bugfix mentioned above?

This has to be fixed for the Yii cache storage driver and then it will be released.

[lenvanessen]

Hi @bencroker,

How is it going with the YiiCache Storage adaptor?

[bencroker]

@lenvanessen My tests are passing using the YiiCacheStorage driver https://github.com/putyourlightson/craft-blitz/commit/d708dccf59b0877a869ed0102629f672f1c3632e

Can you please test again using https://github.com/putyourlightson/craft-blitz/commit/5cef138f7d0a08202cca74fdfe684c44081db976 which may have resolved it?

[bencroker]

Released in version 3.6.9.

[lenvanessen]

Thanks mate, sorry i didn't have the time to test

[hiasl]

@bencroker: I just updated to 3.6.10 (not having installed 3.6.9 before) and I still get 2 directories in the cache dir, when the directory name contains a special char. I understood this was just a workaround, or is this going to stay like this?

I can confirm, that pages in both directories get deleted when the corresponding entry is modified. I can also confirm, that cache warming seems to create the files in both directories.

[bencroker]

This is the fix as it seems that depending on how the server is configured, it will look for the cached file in one of those two directories.