Multi website - customer create form - Attribute Cache not taking account the current website/store

eInyzant commented 7 years ago

Preconditions

Magento CE 2.1.9
Multi website / and multi stores (B2B (with languages : FR, EN) + B2C (with languages FR, EN, JP)
Php 5.6

Steps to reproduce

For the B2b website I want to display Tax / Vat field. And I don't want to display DOB field
For the B2C website I don't want to display Tax / Vat field, but I do want to display DOB field (optional).
So I set up these parameters in the backend Stores => Settings/Configuration => Customers / Customers configuration => Name and Address Options

Website	Setting	Value
B2B	Show Date of birth	No
B2B	Show Tax/VAT Number	Optional
B2C	Show Date of birth	Otptional
B2C	Show Tax/VAT Number	No

Expected result

I flush all cache
I go to B2B website on customer/account/create => generate form and display Tax / Vat field as optional and hide Dob field
I go to B2C website on customer/account/create => generate form and display Dob field as optional but hide Tax / Vat field.

Actual result

I flush all cache
I go to B2B website on customer/account/create => generate form and it display Tax / Vat field as optional and hide Dob field => OK!
I go to B2C website on customer/account/create => generate form and it display the same form as B2B website => NOK!
I flush cache
I go to B2C website on customer/account/create => generate form and it display Dob field as optional but hide Tax / Vat field => OK!
I go to B2B website on customer/account/create => gernerate form and it display the same form as B2C website => NOK!

going deeper in the code I think the problem is related to the AttributeCache class :

Magento\Customer\Block\Widget\Taxvat Magento\Customer\Block\Widget\AbstractWidget::_getAttribute (:96) Magento\Customer\Model\Metadata::getAttributeMetadata (:62) Magento\Eav\Model\Entity\AttributeCache::saveAttributes (:108) => The cache Key does not take the website_id or store_id into account. So once it is cached on a website it will give the same result on the other websites.

orlangur commented 7 years ago

@eInyzant thanks for bug report!

Yeah, this mechanism was quite buggy and thus it was decided to completely eliminate it in 2.2.0: https://github.com/magento/magento2/commit/3ae7c1909daad96d5a0ad6ab7048455448ba7480

This is not something that could be backported due to BC constraints, as a workaround you can try just to disable this cache type.

Hexmage commented 6 years ago

In 2.2.2 the first website loaded, still caches the visibility and requirement for the attribute for all websites.

Magento\Customer\etc\di.xml (:38)

<preference for="Magento\Customer\Api\CustomerMetadataInterface"
                type="Magento\Customer\Model\Metadata\CustomerCachedMetadata" />

Seems to be the issue, the taxvat still gets cached in the customer metadata.

orlangur commented 6 years ago

Hi @Hexmage, are you saying that the issue is still reproduced on 2.2.2 with exactly the same steps?

Hexmage commented 6 years ago

@orlangur yes. But I only specifically tested taxvat.

Hexmage commented 6 years ago

@orlangur And the issue is also present on the customer/account/edit page.

orlangur commented 6 years ago

@Hexmage, thanks, reopening for recheck then.

Hexmage commented 6 years ago

@orlangur I believe the issue is caused by the fact that the taxvat settings are cached in the Customer Metadata.

magento-engcom-team commented 6 years ago

@eInyzant, thank you for your report. We've acknowledged the issue and added to our backlog.

rbayet commented 6 years ago

Hi,

Just adding my 2 cents : I used the method descripted here which works perfectly for the per-website visibility on a customer attribute on the customer/register form.

BUT in 2.2.2 it seems to fails on a customer address attribute (say "telephone" - yes, it requires you to change in DB the "customer_eav_attribute/is_system" to 0 for that attribute to change the settings in admin) according to the page/form you hit first after a cache refresh.

TL;DR version : if you plan to use that method for a customer address attribute, double checks what happens after a cache flush when hitting FIRST on a customer/address/edit page THEN a customer/account/create page and vice versa.

Long version : A bit of context : I tricked by layouting my customer/register form to show the customer address fields, so a customer which register even outside the checkout has to provide both contact info and a default address. I changed the is_required of telephone : required for website/store A, not required for website/store B. I also changed the Magento_CustomerCustomAttributes/templates/customer/form/edit and register.phtml to \Magento\Customer\Block\Widget\Telephone instead of an hard-coded required telephone field.

From a starting point with no cache whatsoever :

if I refresh first the customer/account/create pages on each website/store, it's ok : the 'telephone' customer address attribute is required when it should, not required when it should not
- if I then go on a customer/address/edit in the customer account area, it's ok too : the 'telephone' attr. is required on A, not required on B
BUT if a customer/address/edit page is hit first on website A (where the attr. is required) after the cache clean, then 'telephone' becomes required on the customer/address/edit of website B too
- of course, the customer/account/create form of B is also impacted

I've tried to dig, so far, I suspect something inside \Magento\Customer\Model\AttributeMetadataDataProvider and \Magento\Eav\Model\Config :

\Magento\Customer\Model\AttributeMetadataDataProvider::getAllAttributeCodes injects a store context before calling \Magento\Eav\Model\Config::getEntityAttributeCodes
\Magento\Customer\Model\AttributeMetadataDataProvider::getAttribute does not

\Magento\Eav\Model\Config::getEntityAttributes maintains cache objects which correctly vary according to the current store but also uses/updates a memory cache shared with \Magento\Customer\Model\AttributeMetadataDataProvider::getAttribute (see \Magento\Eav\Model\Config::loadAttributes and \Magento\Eav\Model\Config::saveAttribute and \Magento\Eav\Model\Config::_createAttribute).

The cache key used in \Magento\Eav\Model\Config::getEntityAttributes is built as

$cacheKey = self::ATTRIBUTES_CACHE_ID . '-' . $entityType->getId() . '-' . $storeId . '-' . $attributeSetId;

=> On the customer/account/create page, the code never reaches the last lines of \Magento\Eav\Model\Config::getEntityAttributes with an entity type of 2 (customer address attributes).

        $this->attributesPerSet[$cacheKey] = $attributes;

        return $attributes;
    }

The sequences of cacheKeys on that line for customer/account/create pages are :

3 [store_id] 3
3       0        0

On a customer/account/edit they are :

1 [store_id] 0
2      0         0
2      0         2
2 [store_id] 2
1 [store_id] 1
3 [store_id] 3
3      0         0

The first 3 entries are reached before the Telephone widget is executed (so before the getIsRequired() call). From the look of the call stacks, it's related to the logged-in customer data retrieval (customer info + customer's addresses collection).

Will try to keep digging ...

Regards, RB.

CompactCodeEU commented 6 years ago

Any updates on this issue? This seems like a bug that breaks a lot of the purposes of a multi website enviroment

magento-engcom-team commented 6 years ago

@magento-engcom-team Thank you for verifying the issue. Based on the provided information internal tickets MAGETWO-95133, MAGETWO-95134 were created

CompactCodeEU commented 6 years ago

I have created a module for now that just retrieves the config like any user would expect. I do not see the advantage of loading the whole attribute from the cache to get those settings.

With the use of plugins i overwrite all the isRequired() and isVisible() methods

I also believe that getting the settings provide less load then getting the whole attribute even if it is from the cache

callumstar commented 5 years ago

Hi @magento-engcom-team, is there any update on this being fixed? I am seeing this issue in a 2.2.6 build too.

TenderDishes commented 4 years ago

Hello

TL;DR

I have encountered a similar phenomenon which i believe has the same root cause as this problem here.

For a better structure i am using the issue template.

Preconditions (*)

Magento CE 2.3.4
PHP 7.2
Two stores (B2B and B2C) and for each own websites and view

Steps to reproduce (*)

Go to backend configuration and switch scope to B2B store
Set Customers->Customer Configuration->Create New Account Options->Show VAT Number on Storefront to yes (in config path: customer/create_account/vat_frontend_visibility)
Set Customers->Customer Configuration->Name and Address Options->Show VAT Number on Storefront to required (in config path: customer/address/taxvat_show)
Flush the cache
Goto customer register account on storefront in B2B store
Create a new account
Go back to the backend and open the newly created customer
Try to save the customer (it doesn´t matter if a value gets changed)

Expected result (*)

The account gets saved successfully

Actual result (*)

Error stating that: The "Tax/VAT Number" attribute value is empty

You can solve the problem when you flush the eav ("EAV types and attributes") cache.

What i think happens internally: The attribute gets rendered in the form as required and therefore gets saved in the attribute cache as required. Then when you openen the customer in backend the attribute gets shown as optional (as the default settings state), but the attribute is cached as required. Because there is apperently no checking of the store or website within the cache the validation failed. What I can´t explain is why the value is empty in the first place, because when you check the validation function in code the value is always empty.

My debugged call stack: vendor/magento/module-customer/Controller/Adminhtml/Index/Validate.php:58 vendor/magento/module-customer/Model/AccountManagement.php:1057

kestraly commented 4 years ago

*Preconditions ()** Magento CE 2.3.4 PHP 7.3 Two stores (B2B and B2C) - 1 Website / 1 Store View attached

Same issue, as soon as you open up one store, it caches the settings for that store and the other store has to have the same fields.

Edit: Disable eav cache and problem resolved.

digvijay2017 commented 4 years ago

I have got it the same issue. It's really a serious issue. Some of the customers not able to do the registration. I am strange to see that @m2-community-project added in low priority, How disgusting? Disable the EAV cache is not a solution, it's a serious issue. Magento community must provide the solution for this. It's occurring in Enterprise and Cloud as well. Poor Magento community @magento-engcom-team

dankhrapiyush commented 4 years ago

This issue is reproduced in the Magento 2.3.5-p1.

xeontcs commented 3 years ago

Any solution for this issue? Reproduced in the Magento 2.3.5-p1 and 2.4.2. We need a fast solution for this @magento-engcom-team

makproductions commented 3 years ago

This needs to be a higher priority! issue is happening for us and customer billing address attributes at checkout disappear

mamsincl commented 2 years ago

and still an issue .... just freshly experienced under 2.4.1 ....

amenk commented 2 years ago

Any update on this?

JoostWan commented 2 years ago

Still a issue over here with 2.4.3-p2

stkrelax commented 2 years ago

Due to attributemetadata is added to attribute on load , loaded attributes will have scope_ values set.

First Request after Cache Clean (may be store 4) will save this attribute in Cache via Module Eav Model Config with Cache Key EAV_ENTITY_ATTRIBUTES . $entityTypeCode..... no store provided

Second Request (may be store 3) will load attributes from Eav Config Cache. These loaded Attributes will be used to create AttributeMetadata which of course will be from store 4 and not 3... This generated MetaData than will be saved in MetaDataCache with store_id in cache Key (ATTRIBUTE_METADATA_INSTANCES_CACHEcustomer_addresscompany3)

Third and other Requests will use MetaDataCache which is already wrong filled.

Only Solution atm i found is to split eav Config Cache Website wise (not storewise because customer_eav_attribute_website does not support store, only website )

therefor construct of Eav Model Config added

protected StoreManager $_storeManager;
public function __construct(
...
    StoreManager $storeManager = null
}{
...
    $this->_storeManager = $storeManager ?: ObjectManager::getInstance()->get(StoreManager::class);
}

and cache key changed to $cacheKey = self::ATTRIBUTES_CACHE_ID . $entityTypeCode . $this->_storeManager->getWebsite()->getId();

/**
 * Initialize attributes from cache for given entity type
 *
 * @param Type $entityType
 * @return bool
 */
private function initAttributesFromCache(Type $entityType)
{
    $entityTypeCode = $entityType->getEntityTypeCode();
    $cacheKey = self::ATTRIBUTES_CACHE_ID . $entityTypeCode . $this->_storeManager->getWebsite()->getId();
    if ($this->isCacheEnabled() && ($attributes = $this->_cache->load($cacheKey))) {
        $attributes = $this->serializer->unserialize($attributes);
        if ($attributes) {
            foreach ($attributes as $attribute) {
                $this->_createAttribute($entityType, $attribute);
                $this->_attributeData[$entityTypeCode][$attribute['attribute_code']] = $attribute;
            }
            return true;
        }
    }
    return false;
}

Also saving of cache extended with website_id

    if ($this->isCacheEnabled()) {
        $this->_cache->save(
            $this->serializer->serialize($this->_attributeData[$entityTypeCode]),
            self::ATTRIBUTES_CACHE_ID . $entityTypeCode . $this->_storeManager->getWebsite()->getId(), 
            [
                \Magento\Eav\Model\Cache\Type::CACHE_TAG,
                \Magento\Eav\Model\Entity\Attribute::CACHE_TAG
            ]
        );
    }

    \Magento\Framework\Profiler::stop('EAV: ' . __METHOD__);
    return $this;
}

aadmathijssen commented 1 year ago

Got bitten by this this week on Adobe Commerce 2.4.4-p2. At first I thought this was related to the custom customer attributes module, but it turned out I could easily reproduce this on Magento Open Source using the standard fax attribute.

Here are the steps I took in a multistore setting with website A and B on both Magento Open Source and Adobe Commerce:

In the admin, go to Stores > Configuration > Customers > Customer Configuration. Change the scope from Default Config to website A, and under Name and Address Options, change "Show Fax" from "No" to "Optional" and click "Save Config."
Go to the website A frontend, create an account, and click on the address tab to open an address form. Observe there is a "fax" input field.
Clear the cache, and open the website B frontend. Now go back to website A, open the address form once again and observe the "fax" input field has disappeared.

Minirock commented 1 year ago

Looks like this issue is still there in 2.4

Website A : Tax vat optionnal Website B : Taxvat required

The show in create form seems to work. But the is required is not working. That means even if the field is required in website B it will be show as not mandatory in frontend.

Second part of the issue, even if your customer has the taxvat filled, when you go to the admin, it will tell you that the taxvat field is still required.

For the admin part I have been digging onto it, it seems to be related to this method

    /**
     * Extract data from request and return associative data array
     *
     * @param \Magento\Framework\App\RequestInterface $request
     * @param string $scope the request scope
     * @param boolean $scopeOnly search value only in scope or search value in global too
     * @return array
     */
    public function extractData(\Magento\Framework\App\RequestInterface $request, $scope = null, $scopeOnly = true)
    {
        $data = [];
        foreach ($this->getAllowedAttributes() as $attribute) {
            $dataModel = $this->_getAttributeDataModel($attribute);
            $dataModel->setRequestScope($scope);
            $dataModel->setRequestScopeOnly($scopeOnly);
            $data[$attribute->getAttributeCode()] = $dataModel->extractValue($request);
        }
        return $data;
    }

For some reason the taxvat field which is present in the request object is not kept in the data object when it should be. An even stranger behaviour is when i tried to log into this method. If I var dump die $data in this method i do have the taxvat attribute.

But if I die a little after in the validate

 $data = $customerForm->extractData($this->getRequest(), 'customer');

The result is inconsistent, sometimes the taxvat seems to be there, sometimes it's not.

This is obviously something related to cache and multistore for this configuration, but I can't get through it.

So far i only see 2 workarounds. Getting 2 template for the create customer form in frontend with the b2b website having the taxvat field mandatory without condition of isRequired as this one is not working properly. And in backend go to the exception thrown and in the case of the taxvat make myself a direct request to check is the taxvat is really empty or not. This is really a bug that should be fixed since the time it's alive cause it's kinda of a critical feature to my eyes. I don't understand how this can still be not fixed since reported in 2017.

Did I missed any work around in this thread ?

kestraly commented 1 year ago

Can confirm it’s still an issue. I’ve left the EAV cache off still until it’s fixed.

Minirock commented 1 year ago

Sadly in my case we are almost dealing with big data so it's not a solution i can envisage to disable eav cache

m2-assistant[bot] commented 6 months ago

Hi @engcom-Delta. Thank you for working on this issue. Looks like this issue is already verified and confirmed. But if you want to validate it one more time, please, go though the following instruction:

1. Add/Edit Area: XXXXX label to the ticket, indicating the functional areas it may be related to.
1. Verify that the issue is reproducible on 2.4-develop branch
  Details
  - Add the comment @magento give me 2.4-develop instance to deploy test instance on Magento infrastructure.
  - If the issue is reproducible on 2.4-develop branch, please, add the label Reproduced on 2.4.x.
  - If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!
1. If the issue is not relevant or is not reproducible any more, feel free to close it.

Join Magento Community Engineering Slack and ask your questions in #github channel.

engcom-Delta commented 5 months ago

Hi @eInyzant ,

Thanks for your reporting and collaboration.

We have verified the issue in latest 2.4.7 and 2.4-develop instance and the issue is no more reproducible.Kindly refer the attached screenshots.

Steps to reproduce:-

1.Create a new Website, Store and Store View 2.Go to Stores->Configuration->Customers->Customer Configuration->Name and Address Options 3.Switch the scope Main Website and set Show DOB field to No and Show TAX/VAT Number to Optional and save config 4.Now switch the scope to Second Website and set Show DOB field to Optional and Show TAX/VAT Number to No and Save config 5.Enable Store code in URL's, switch to Default scope and go to Stores->Configuration->General->Web->Url Options->Add Store Code to Urls and set to Yes and then Save Config 6.From System->Cache Management Flush Cache Storage 7.Go to Main Website Storefront Create Account Page 8.Go to https://yourmagento.com/default/customer/account/create/ 9.We see Tax/Vat Number field and don't see DOB field 10.Then go to new created website Create Account Page 11.Go to https://yourmagento.com/new/customer/account/create/ 12.We see DOB field and don't see Tax/Vat Number field

Websites created