FPC Cache via Files or Redis does not work on Magento 2.4.7 for multi-view stores

[dandrikop]

Preconditions and environment

This bug report is actually related with the issue: https://github.com/magento/magento2/issues/38626

Besides the Files used as the default medium for FPC, I tested Redis with Magento 2.4.7-p2 and FPC did not work. Namely it seems that cached content is stored at Files or Redis. The problem is that Magento cannot somehow identify the cached pages so it thinks that they have not been cached.

This problem is No-Go for Magento 2.4.7 for store owners with multi-domain and multi-view Magento installations, as Magento without FPC is out-of-question. It is said that the problem will be fixed at Magento 2.4.8. However, a patch should become available at Magento Quality Patches; otherwise store owners with multi-domain and multi-view Magento installations should stay at version 2.4.6 till 2.4.8 is released.

Steps to reproduce

1. Set the multi-view store to Developer mode.
2. Visit various pages on the front-end.
3. Monitor the value of the HTTP header "X-Magento-Cache-Debug".

Expected result

The value of the HTTP header "X-Magento-Cache-Debug" is HIT for subsequent requests of the same page.

Actual result

The value of the HTTP header "X-Magento-Cache-Debug" is always MISS no matter how many times a page has been visited at the front-end.

Additional information

No response

Release note

No response

Triage and priority

• [X] Severity: S0 - Affects critical data or functionality and leaves users without workaround.
• [ ] Severity: S1 - Affects critical data or functionality and forces users to employ a workaround.
• [ ] Severity: S2 - Affects non-critical data or functionality and forces users to employ a workaround.
• [ ] Severity: S3 - Affects non-critical data or functionality and does not force users to employ a workaround.
• [ ] Severity: S4 - Affects aesthetics, professional look and feel, “quality” or “usability”.

[m2-assistant[bot]]

Hi @dandrikop. Thank you for your report. To speed up processing of this issue, make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce.

• For more details, review the Magento Contributor Assistant documentation.
• Add a comment to assign the issue: @magento I am working on this

• To learn more about issue processing workflow, refer to the Code Contributions.

  Join Magento Community Engineering Slack and ask your questions in #github channel. :warning: According to the Magento Contribution requirements, all issues must go through the Community Contributions Triage process. Community Contributions Triage is a public meeting. :clock10: You can find the schedule on the Magento Community Calendar page. :telephone_receiver: The triage of issues happens in the queue order. If you want to speed up the delivery of your contribution, join the Community Contributions Triage session to discuss the appropriate ticket.

[dandrikop]

It is absolutely necessary to have patch available at Magento quality patches for this issue:

https://experienceleague.adobe.com/tools/commerce-quality-patches/index.html

This is a serious problem, as FPC is a crucial performance factor for Magento.

[m2-assistant[bot]]

Hi @engcom-Hotel. Thank you for working on this issue. In order to make sure that issue has enough information and ready for development, please read and check the following instruction: :point_down:

• [ ] 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
• [ ] 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue.
• [ ] 3. Add Area: XXXXX label to the ticket, indicating the functional areas it may be related to.
• [ ] 4. Verify that the issue is reproducible on 2.4-develop branch
  Details

  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!
• [ ] 5. Add label Issue: Confirmed once verification is complete.
• [ ] 6. Make sure that automatic system confirms that report has been added to the backlog.

[engcom-Hotel]

Hello @dandrikop,

Thanks for the report and collaboration!

We are discussing this issue internally, meanwhile please use the below commits to fix this issue:

https://github.com/search?q=repo%3Amagento%2Fmagento2+AC-11819&type=commits

We are moving this issue On Hold.

Thanks

[engcom-Hotel]

Hello @dandrikop,

We have tried to reproduce the issue in 2.4.7-p3 with both default FPC and Redis. and for us the issue is not reproducible. We have created multiple store views to reproduce the issue.

The first time when we access a store view it shows as Miss and the next time it shows as Hit. Please refer to the below screenshot for reference:

English Storeview

French Storeview

Please let us know if we have missed anything.

Thanks

[dandrikop]

The error depends on the implementation. My multi-store implementation has 7 websites, each website has one store, and each store has one store view. The routing at the appropriate store view is regulated by the .htaccess file with directives such as the following ones:

SetEnvIf Host fr.dlastore.com MAGE_RUN_CODE=french_dla SetEnvIf Host fr.dlastore.com MAGE_RUN_TYPE=store

This problem has been reproduced and identified with AC-11819. My concern is that we'd better have a patch available at:

https://experienceleague.adobe.com/tools/commerce-quality-patches/index.html

[engcom-Hotel]

Hello @dandrikop,

Thanks for the input.

We have tried to reproduce the issue with the multi-websites and multi-stores, and the issue has been reproducible for us. Let me keep this issue on hold to discuss it with the PO.

We will keep you posted.

Thanks

[engcom-Hotel]

Hello @dandrikop,

Could you please confirm if you are running the above-mentioned scenario (multi-store, multi-website) with Varnish configured? If so, what is the value of the x-magento-cache-debug header?

[dandrikop]

Hello @engcom-Hotel ,

I'm running the above scenario with Files for Magento FPC. My app/etc/env.php file contains the below code for the section 'cache':

'cache' => [ 'graphql' => [ 'id_salt' => 'i38cdiM3Hdrq2PKAPJ5NQNyUbZkSqCUe' ], 'frontend' => [ 'default' => [ 'id_prefix' => '8e8_' ], 'page_cache' => [ 'id_prefix' => '8e8_' ] ], 'allow_parallel_generation' => false ]

The HTTP header x-magento-cache-debug was always MISS, though I could find the relevant files under the directory var/page_cache/mage--*. This gave me the impression, that the content was indeed cached, but somehow it was not fetched from the Magento FPC when it was requested again. To be more precise, the behavior was as follows:

1. I visited a page of my store and noticed that the HTTP header x-magento-cache-debug=MISS.
2. I found the generated file var/page_cache/mage--0/mage---8e8_6B4147C5910C9E127263E348A729BC033906CBBF and noted its timestamp.
3. I revisited the page and noticed again that the HTTP header x-magento-cache-debug=MISS.
4. I rechecked the file var/page_cache/mage--0/mage---8e8_6B4147C5910C9E127263E348A729BC033906CBBF and its timestamp was updated.

I also tried the same scenario with Redis Magento FPC, and the result was the same as with Files. I have not tested the scenario with Varnish. This issue is supposed to be resolved at Magento 2.4.8, but this is a serious bug that needs to be resolved as soon as possible, given that under these circumstances we cannot upgrade to Magento 2.4.7. I recommend that you issue a patch published at:

https://experienceleague.adobe.com/tools/commerce-quality-patches/index.html

[engcom-Hotel]

Keeping this issue On hold till further updates.

[MartinL1983]

Hello!

We encountered this issue yesterday on our staging environment and did some investigation. We discovered a difference between the results returned by $this->identifier->getValue() and $this->identifierForSave->getValue() in PageCache/Kernel.php.

Here are the values being returned:

• $this->identifierForSave->getValue() = 31cf127754c679b1a38b4a2c19af6f9ad2b3d849
• $this->identifier->getValue() = MAGE_RUN_TYPE=store|MAGE_RUN_CODE=aedwinkel_nl|31cf127754c679b1a38b4a2c19af6f9ad2b3d849

It’s clear that one of the methods is returning an unexpected value. We’re not yet certain which one. Logically, it seems the hashed version is the correct value, but we are also unsure if MAGE_RUN_TYPE and MAGE_RUN_CODE are meant to be part of the hash.

Since it’s surprising that an issue this small hasn’t been identified by others, we are continuing to investigate to confirm if this is indeed the root cause.

For now, we’re using the provided patch and haven’t observed any negative effects while doing so. kernel.patch.txt

We’ll post any new information we find here.

Does anybody else have any ideas to confirm if we are on the right track?

[dandrikop]

@MartinL1983

Indeed, this problem has been reported with https://github.com/magento/magento2/issues/38626. What I added was just some tests with REDIS used for FPC in addition to Files FPC for which the problem was initially reported. I don't know if the problem also occurs if Varnish is used for FPC.

This problem has been routed to be resolved with Magento 2.4.8 version. See AC-11819 at the below link:

https://experienceleague.adobe.com/en/docs/commerce-operations/release/notes/magento-open-source/2-4-8

However, I believe that it should be resolved, as soon as possible, with an official patch for the version 2.4.7 in which it appeared for the first time. It is obvious that multi-stores which are currently on Magento 2.4.6 cannot be upgraded to 2.4.7 due to this serious problem, as a multi-store on Magento 2.4.7 actually works without FPC, at least if FPC is configured with either Files or REDIS.

[Webroju]

@MartinL1983, thanks for the patch, but I see no positive effect from the patch. (NGINX, MAGE_RUN_CODE)

[MartinL1983]

@Webroju

Did you add it to your composer.json followed by a composer install?

[Webroju]

@MartinL1983 I have not installed it over composer, I have installed like a patch. patch -p1 < filename.patch

[MartinL1983]

@Webroju

I have never applied a patch that way, so I could not tell you what is wrong.

I always apply patches via composer like this:

"extra": {
    "magento-force": "override",
    "composer-exit-on-patch-failure": true,
    "patches": {
        "magento/framework": {
            "Patch for FPC Identifier on load": "patches/composer/magento/framework/cache/kernel.patch"
        },
    }
}

[dandrikop]

@MartinL1983, @Webroju

I believe that an official patch should become available at the below page:

https://experienceleague.adobe.com/tools/commerce-quality-patches/index.html

I don't think that there is only a small number of multi-store Magento installations around the world. The reason that this issue has not taken significant publicity yet maybe that either a few installations may be upgraded to 2.4.7, or some store owners who have already upgraded to 2.4.7 may have not yet realized that they don't actually have FPC.

[dandrikop]

@MartinL1983,

The attributes MAGE_RUN_TYPE and MAGE_RUN_CODE must be part of the hash. I'm using a 3rd party module for Full Page Cache warming on Magento 2.4.6, where the FPC works fine with Magento multi-store, and I can see that the Vary data shown by that module includes the MAGE_RUN_TYPE and MAGE_RUN_CODE. I attached here in a relevant screenshot from Magento 2.4.6:

Are you sure that the patch you mention can work with the FPC without any complications in other parts of Magento functionality? I'm just asking because the below patch that was initially issued for this problem includes a lot of changes:

https://github.com/search?q=repo%3Amagento%2Fmagento2+AC-11819&type=commits

On the other hand, this very patch was never included in Magento Quality Patches for Magento 2.4.7.