The shipping information was unable to be saved. verify the input data and try again.

mehmetuygun commented 5 years ago

Additional Details

Detailed explanation and result of the investigation https://github.com/magento/magento2/issues/23908#issuecomment-538087658

Preconditions (*)

Magento 2.3.2, 2.3-develop

Steps to reproduce (*)

Create a customer account
Go to my account, and add a shipping address 2.5. Enable Persistent Shopping Cart (updated Oct 10.2019)
Add any product and ~go to the checkout~ !Impotent: wait until the customer session expires (one hour by default) but no longer than Persistence Lifetime is set to (updated Oct 10.2019) 3.5 Reload the page so that somewhere in the upper-right corner You get "Not You?" link next to the welcome message (updated Oct 10.2019) 3.6 Clik on "Not You?" enter the credentials of the customer and log in (updated Oct 10.2019) 3.6 Go to Mini Cart end press Precede to Checkout (updated Oct 10.2019)
Select the address you have just added
Select a shipping method and click next button

Expected result (*)

Payment page should show up

Actual result (*)

m2-assistant[bot] commented 5 years ago

Hi @mehmetuygun. Thank you for your report. To help us process this issue please make sure that you provided the following information:

[ ] Summary of the issue
[ ] Information on your environment
[ ] Steps to reproduce
[ ] Expected and actual results

Please make sure that the issue is reproducible on the vanilla Magento instance following Steps to reproduce. To deploy vanilla Magento instance on our environment, please, add a comment to the issue:

@magento give me 2.3-develop instance - upcoming 2.3.x release

For more details, please, review the Magento Contributor Assistant documentation.

@mehmetuygun do you confirm that you were able to reproduce the issue on vanilla Magento instance following steps to reproduce?

[ ] yes
[ ] no

Join Magento Community Engineering Slack and ask your questions in #github channel.

m2-assistant[bot] commented 5 years ago

Hi @engcom-Bravo. Thank you for working on this issue. In order to make sure that issue has enough information and ready for development, please read and check the following instruction: :point_down:

[ ] 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
Details
If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.
[ ] 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.
[ ] 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.
[ ] 4. Verify that the issue is reproducible on 2.3-develop branch
Details
- Add the comment @magento give me 2.3-develop instance to deploy test instance on Magento infrastructure.
- If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
- If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!
[ ] 5. Verify that the issue is reproducible on 2.2-develop branch.
Details
- Add the comment @magento give me 2.2-develop instance to deploy test instance on Magento infrastructure.
- If the issue is reproducible on 2.2-develop branch, please add the label Reproduced on 2.2.x
[ ] 6. Add label Issue: Confirmed once verification is complete.
[ ] 7. Make sure that automatic system confirms that report has been added to the backlog.

mehmetuygun commented 5 years ago

@magento give me 2.3.2 instance

magento-engcom-team commented 5 years ago

Hi @mehmetuygun. Thank you for your request. I'm working on Magento 2.3.2 instance for you

magento-engcom-team commented 5 years ago

Hi @mehmetuygun, here is your Magento instance. Admin access: https://i-23908-2-3-2.instances.magento-community.engineering/admin Login: admin Password: 123123q Instance will be terminated in up to 3 hours.

engcom-Bravo commented 5 years ago

Hello @mehmetuygun. Thanks for applying. We've been not able to reproduce Your issue on a vanilla magento instance. Please confirm You are able to to so.

mehmetuygun commented 5 years ago

I couldn't reproduce issue on magento instance either. After I adding $customerId = $this->customerSession->getId(); in to begging of the function private function doValidate(AddressInterface $address, ?int $customerId): void in the file magento/module-quote/Model/QuoteAddressValidator.php the error stop showing up. And I could pass to payment page. How is possible customerId passing as null? it happens when I logged in and trying place order with saved address.

engcom-Bravo commented 5 years ago

@mehmetuygun, if an issue is not reproduced on a fresh magento instance, free from any custom code, third party extensions, etc., then it is not a core magento bug this repository is intended for reporting. Please refer to the Community Forums or the Magento Stack Exchange site for advice or general discussion about this issue. Thus we are supposed to close this issue but please feel free to reopen it if deem necessary. Thanks for applying.

ron-jomashop commented 5 years ago

This issue is related to this issue: https://github.com/magento/magento2/issues/23618

onepack commented 5 years ago

I created https://github.com/magento/magento2/issues/23618# and there is a reference to this issue. So I check and the first I see is that this issue is closed here as well. I understand that you are eager to close issues to keep numbers low but clearly this issue can have a huge impact: Customers can not checkout. Is there anyone from the magento core team that is willing to take time to find out what is the root of this issue as it is not entirely clear why people are getting this message.

sbrendtro commented 5 years ago

We've been seeing this as well on a 2.3.2 install. Our data was imported from a Magento 1.9.x install using the Magento Data Migration Tool. We were originally suspecting something to do with the data migration, but it still happens when you create a new account. At the moment we're disabling our 3rd party modules one by one to see if we can narrow it down, but this behavior seems to have been around since before we added our 3rd party modules.

onepack commented 5 years ago

I imported the customer data via CSV as I have bad experiences with the data migration tool. The account that had problems was a testing account. So I deleted it and re-created it. But it happened again after that with the new testing account (with the same email).

sbrendtro commented 5 years ago

Disabled all third party modules and still seeing same behavior. Traced things back through the code and found some interesting things happening:

exception.log shows the error coming from [2019-08-06 22:05:07] main.CRITICAL: Invalid customer address id 160022 {"exception":"[object] (Magento\Framework\Exception\NoSuchEntityException(code: 0): Invalid customer address id 160022 at .... vendor/magento/module-quote/Model/QuoteAddressValidator.php:79)"}

That's QuoteAddressValidator::doValidate(AddressInterface $address, ?int $customerId), which is being called by line 130 in the same file. The second parameter for doValidate is set as: $cart->getCustomerIsGuest() ? null : $cart->getCustomer()->getId()

However, $cart->getCustomerIsGuest() is reporting me as a guest, when in fact I'm not. I logged in during checkout, and I'm continuing to refresh the checkout page, so it should be pretty apparent by now that I'm logged in. It is actually showing me all of my addresses, so it clearly thinks I'm logged in.

Ultimately, getCustomerIsGuest() pulls the information from the quote table, which I checked and found that for my customer_id, my active quote had customer_is_guest column set to 1. I'm not sure how the quote (merged from my guest cart) ended up with customer_is_guest still set to 1, but after clearing it, the problem vanished (at least for now).

The following query helped me find the other quotes in the same situation:

SELECT entity_id, customer_id FROM quote WHERE customer_id != 0 AND customer_is_guest = 1;

Then I reset them to the proper status:

UPDATE quote SET customer_is_guest = 0 WHERE customer_id != 0 AND customer_is_guest = 1;

The only quotes I could find in this state belonged to test accounts. I'll continue with testing with 3rd party modules enabled, and comment back if I find anything else improperly messing with the customer_is_guest field.

onepack commented 5 years ago

@sbrendtro, this is a good catch! I was not getting this critical message. But I was having the same in the quote table customer_is_guest was 1 with this testing customer. Setting it to 0 fixed things. The root cause is yet unclear. I will keep an eye on this.

ghost commented 5 years ago

@sbrendtro

Do you mean

UPDATE quote ...

rather than

UPDATE entity ...

?

sbrendtro commented 5 years ago

@sbrendtro

Do you mean

UPDATE quote ...

rather than

UPDATE entity ...

?

Yes, thanks for pointing that out. Just edited my comment.

ghost commented 5 years ago

No, thank you for pointing out the fix for what was turning out to be a massive pain in the backside for a client!

ron-jomashop commented 5 years ago

@onepack I think you should put this solution in your issue too because I believe it will help many people including me.

mertgokceimam commented 5 years ago

This is an issue that effects both Magento 2.2.9 and 2.3.2

It is also pretty disturbing to see Magento team denies that such issue exist. I can also confirm that the issue occurs on a clean installation.

Thanks to @sbrendtro we corrected the issue on our shop but denying such important issues on an Enterprise solution is not acceptable on magento teams behalf.

engcom-Bravo commented 5 years ago

Dear members of this conversation. Please provide the steps following which we can reproduce the issue on a vanilla Magento installation of 2.3-develop version - the issue will be immediately reopened and confirmed. Dear @onepack, we are not eager to close the issues. We just follow our workflow (please see details to step 4 of this comment above https://github.com/magento/magento2/issues/23908#issuecomment-515489471). Thank You all for understanding.

mertgokceimam commented 5 years ago

It is really hard to reproduce as this is not an existing issue for everyone. For example when i checked our production instance , we had 85 total user quote that was effected by the issue. We of course corrected the quote issue but a step by step way to reproduce it is extremely hard.

One day after correcting the quote's we have this issue back in place. mysql> SELECT entity_id, customer_id FROM quote WHERE customer_id != 0 AND customer_is_guest = 1; +-----------+-------------+ | entity_id | customer_id | +-----------+-------------+ | 57600 | 313840 | | 24340 | 397145 | | 106394 | 483487 | | 117654 | 483683 | +-----------+-------------+

Magento seriously urgently release a patch for this issue as this is effecting many customers incomes and most are not possibly aware of the issue.

Just because Magento team choose to close close issues for keeping clean board doesn't mean that us Enterprise customers are going to loose money.

In no condition Magento should allow a Logged in customer quote assigned as a Guest customer. But this can happen because of . a weird logic Magento team choose to implement. This shouldn't happen at all.

tuyennn commented 5 years ago

@engcom-Bravo We have same problem with magento since the 1st day of launching. The magento team should take a look on how you implement quote validation with customer_is_guest on quote and care how it's merging.

sbrendtro commented 5 years ago

Here's a temporary workaround at the database level (USE WITH CAUTION). Since no one seems to be able to figure out where this is coming from, I figured I would use a MySQL trigger to prevent customer_is_guest from being set anything but 0 when customer_id is set. You'll want to do your own testing before putting this in production. But I can't think of any real risk, as customer_is_guest should really never be set to true for a logged-in customer.... right?

DROP TRIGGER IF EXISTS quote_before_insert_fix_guest;
DROP TRIGGER IF EXISTS quote_before_update_fix_guest;

DELIMITER $$

    CREATE TRIGGER quote_before_insert_fix_guest BEFORE INSERT ON quote
    FOR EACH ROW BEGIN
      IF (NEW.customer_id != 0 AND NEW.customer_is_guest != 0 ) THEN
            SET NEW.customer_is_guest = 0;
      END IF;
    END$$

    CREATE TRIGGER quote_before_update_fix_guest BEFORE UPDATE ON quote
    FOR EACH ROW BEGIN
      IF (NEW.customer_id != 0 AND NEW.customer_is_guest != 0 ) THEN
            SET NEW.customer_is_guest = 0;
      END IF;
    END$$

DELIMITER ;

edited: Both triggers were on BEFORE UPDATE, changed the first one to BEFORE INSERT.

mertgokceimam commented 5 years ago

@sbrendtro i will like to ask a question , do you also use Redis for session management ?

tuyennn commented 5 years ago

@mertgokceimam has same problem while using redis caching the session. Well, any investigation?

sbrendtro commented 5 years ago

No redis, we are using DB session storage.

On Mon, Aug 19, 2019, 10:58 AM mertgokceimam notifications@github.com wrote:

@sbrendtro https://github.com/sbrendtro i will like to ask a question , do you also use Redis for session management ?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/magento/magento2/issues/23908?email_source=notifications&email_token=AAF4V3AUQDYJ7BN5UCN5XCLQFK7LJA5CNFSM4IHEVPE2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4TN3RQ#issuecomment-522640838, or mute the thread https://github.com/notifications/unsubscribe-auth/AAF4V3EM4Y4ACSN3SB7R6V3QFK7LJANCNFSM4IHEVPEQ .

onepack commented 5 years ago

Same here, no redis session caching.

angelomaragna commented 5 years ago

I've been able ro replicate a weird issue, which may be realted to this one, but don't know the correct frontend steps to get this to happen.

You need:

persistent cart enabled and an expired cart for your account
a current quote as guest, with customer_is_guest set a 1

once you get to the checkout and you log in, your cart is merged.

At that point, when you try to remove an item from the cart, the QuoteAddressValidator will always fail.

I've replaced the final function validateForCart with a temporary, and it does allow now to remove items from the cart.

 public function validateForCart(CartInterface $cart, AddressInterface $address): void
    {
        $customer = $cart->getCustomer();
        $customerId = $customer ? $customer->getId() : null;
        $this->doValidate(
                $address,
                        $customerId
                );
    }

onepack commented 5 years ago

This is a great find and fits in the way I was working. As I was testing I would always have products in the cart and would be busy coding or do other stuff on the backend. This would result in a session expiration so I would be logged out. Logging in again would create this issue with my test user. But this could also happen in real-life situations when customers get logged out and will log in again in order to checkout.

santhoshnsscoe commented 5 years ago

Steps to reproduce:

Add products to cart
Go to checkout
Add email id and login from the checkout page
After login, click the next to go to the payment page.
Error will popup with "The shipping information was unable to be saved. Verify the input data and try again."

When we check the quote table.

SELECT entity_id, customer_id FROM quote WHERE customer_id != 0 AND customer_is_guest = 1;
+-----------+-------------+
| entity_id | customer_id |
+-----------+-------------+
|        28 |           4 |
+-----------+-------------+

tcmax76 commented 5 years ago

We're experiencing this on our sites, as well. The database solution @sbrendtro fixed our problem accounts for now, but we'd really like a more permanent solution.

engcom-Bravo commented 5 years ago

@santhoshnsscoe, do I miss something: order ? Thank You.

tcmax76 commented 5 years ago

We had three more records show up in the quote table since Saturday marked with 1 for "customer_is_guest" but also had a customer_id entered (which seems to cause the error). I don't know if this makes a difference, but all three of them had more than an hour between the quote's 'created_at' datetime and the quote's 'updated_at' datetime.

tcmax76 commented 5 years ago

Actually, there seems to be a correlation between these quotes that are erroring and having the 'is_persistent' column checked. We don't have a whole lot of quotes marked 'is_persistent', but all of the ones that include a customer_id while also marked 1 for 'customer_is_guest' (ie, the ones that seem to be causing this error) are marked as 'is_persistent'.

santhoshnsscoe commented 5 years ago

@santhoshnsscoe, do I miss something: ? Thank You.

I checked with base Magento installation, it is working fine. So, there is some problem in either external modules or some settings. Will test completely and will update here.

mertgokceimam commented 5 years ago

Make sure to enable persistant cart so you can re produce it.

webusat commented 5 years ago

@sbrendtro I had the same issue and changing customer_is_guest to a 0 solved it for me. Seems to be very sporadic and happened to two users. It also seems related to the persistent cart settings.

hostep commented 5 years ago

We had to debug this issue as well today on a 2.3.1 shop, thanks to all the helpful comment above and our investigation, this is a brief summary:

The issue here is that a logged in customer still can have the flag customer_is_guest set to 1 on the quote
This flag is the root cause of the bug reported here, although it's not clear yet how this flag is being set incorrectly
We've searched through various shops we maintain ranging between Magento 2.1, 2.2 and 2.3 versions and all versions are affected by this inconsistent customer_is_guest flag on the quotes (we've seen shops where hundreds of quotes existed where this is incorrect)
We believe the bug reported here only came to the surface in Magento 2.3.1 an 2.3.2 due to some changes done recently in the QuoteAddressValidator class. Before those versions, the incorrect flag also existed but didn't really seem to have caused issues.

Our current solution, was to add an after plugin to the getCustomerIsGuest function of the Magento\Quote\Model\Quote class. And do something like this:

    public function afterGetCustomerIsGuest(QuoteModel $subject, $result)
    {
        $customer = $subject->getCustomer();

        return $customer->getId() === null;
    }

So instead of checking the flag from the database, we check if the customer's id is null or not to decide if a customer is a guest or not.

Now, this is only a workaround. To fix this properly in Magento, two things need to happen in my opinion:

the root cause of this bug needs to be found and fixed (https://github.com/magento/magento2/issues/24736 might be related)
a database upgrade script should be written which fixes all incorrect customer_is_guest flags in the quote table, so all existing quotes which are being prevented to go through the checkout start working again.

I'm reopening this issue for now again (yes, I realise we still don't have steps to reproduce, but the issue seems to be very real).

m2-assistant[bot] commented 5 years ago

Hi @engcom-Bravo. Thank you for working on this issue. In order to make sure that issue has enough information and ready for development, please read and check the following instruction: :point_down:

[ ] 1. Verify that issue has all the required information. (Preconditions, Steps to reproduce, Expected result, Actual result).
Details
If the issue has a valid description, the label Issue: Format is valid will be added to the issue automatically. Please, edit issue description if needed, until label Issue: Format is valid appears.
[ ] 2. Verify that issue has a meaningful description and provides enough information to reproduce the issue. If the report is valid, add Issue: Clear Description label to the issue by yourself.
[ ] 3. Add Component: XXXXX label(s) to the ticket, indicating the components it may be related to.
[ ] 4. Verify that the issue is reproducible on 2.3-develop branch
Details
- Add the comment @magento give me 2.3-develop instance to deploy test instance on Magento infrastructure.
- If the issue is reproducible on 2.3-develop branch, please, add the label Reproduced on 2.3.x.
- If the issue is not reproducible, add your comment that issue is not reproducible and close the issue and stop verification process here!
[ ] 5. Add label Issue: Confirmed once verification is complete.
[ ] 6. Make sure that automatic system confirms that report has been added to the backlog.

engcom-Bravo commented 5 years ago

Hi everyone. Incredible but I've just got reproduced the issue on a (n almost) vanilla. Need some additional recheck - not yet sure for 100% I'll have it reproduced once more. But anyway I'll be back with the results.

engcom-Bravo commented 5 years ago

Updated Steps to reproduce. In fact, as I see now these steps to reproduce are scattered throughout the conversation. So theoretically I might have already gotten them if I were a bit more attentive (I bring my apologies for not being that) and it wouldn't have taken me that much time to come to it by myself. It seems to me I managed to reproduce the issue with slightly other steps as well but not regularly. The current steps will got the issue reproduced in any case.

magento-engcom-team commented 5 years ago

@engcom-Bravo Thank you for verifying the issue.

Unfortunately, not enough information was provided to acknowledge ticket. Please consider adding the following:

[ ] Add "Reproduced on " label(s) to this ticket based on verification result

Once all required information is added, please add label "Issue: Confirmed" again. Thanks!

magento-engcom-team commented 5 years ago

:white_check_mark: Confirmed by @engcom-Bravo Thank you for verifying the issue. Based on the provided information internal tickets MC-21568 were created

Issue Available: @engcom-Bravo, You will be automatically unassigned. Contributors/Maintainers can claim this issue to continue. To reclaim and continue work, reassign the ticket to yourself.

hostep commented 5 years ago

Great @engcom-Bravo !

Although, we've seen this problem happening on shops where the persistent cart functionality was never enabled, so there might be other ways to trigger this bug as well. But it's good to already have one case documented properly 👍

vijay-prevaj commented 5 years ago

Is Anyone reproduce the issue in Magento-2.3.2 ?

Please List the steps to reproduce it.

engcom-Bravo commented 5 years ago

Hi, @vijay-prevaj. Have You already tried out these ones?

diode-dynamics commented 5 years ago

I'm not sure if this is appropriate, but can anyone provide further details of any workarounds that have been implemented right now? We are experiencing this issue on a daily basis, and it is tedious to run the script regularly as provided above.

I see that someone added a plugin to a class, but I'm not sure how to implement that. If anyone has specific code I could add in anywhere until this is resolved, I would sincerely appreciate it. Thank you.

hostep commented 5 years ago

@diode-dynamics: A colleague of mine wrote the following patch, which you can apply using https://github.com/cweagans/composer-patches/ to the magento/module-quote composer package, this seems to work in our case.

Be aware that this is only a workaround, not a fix for the problem, the flag customer_is_guest in the database will still be incorrect from time to time, the patch simply causes that flag to be not used and determines the fact if a customer is a guest or not based on the customer id.

diode-dynamics commented 5 years ago

@hostep thank you so much! Understood that this is only a workaround. I've applied the patch, so far it seems to be working. This is a huge help for us until this bug is fixed. Thanks again.

BenItt commented 5 years ago

@hostep we are facing the same strange problem here. Unfortunately your Patch Link is leading to an empty page. Could you possibly repost your patch? That would be great. Thanks so much in advance

magento / magento2