Cart / Checkout performance issue for multiple products in basket

[groomershop]

Todo

• [ ] Analyze where the problem comes from
• [ ] Find the appropriate solution and re-qualify the issue

Describe the bug After adding for example 50 diffrent products to cart (we are a B2B wholesaler, so it's typical for us) the load time of any site (with product, catalog, adding next product, going to checkout) increases significantly (from 200-300ms to about 800ms - on clean install at empty 6-core server).

We have turned profiling and the problem is with amount of MySQL queries, which for 50 products raise to over 2000 and query time about 250ms.

To Reproduce

1. Clean install PrestaShop 1.7.6.
2. Put sample products.
3. Add diffrent products to basket - from 20-30 the speed goes down, about 50 it's slow, when more is going dramitaly slow.
4. Turn on profilling and you will see how much the queries number has increased.

Screenshots Screenshot from profiler in attachment

Additional information PrestaShop version: 1.7.6 PHP version: 7.2.20

[matks]

Hello everyone I notice the latest comments might drive this conversation to a topic not related to the initial issue. In order to avoid further issues I hid the initial post and I request that next comments focus on the initial Cart / Checkout performance issue for multiple products in basket topic 👍

[kpodemski]

@davidcylke I suggest you deleting your last message because it's not clear to whom you wrote it, previous comments were probably deleted by an author

If you are looking for a developer there a better places to do that, official forum, websites for freelancers, etc.

[davidcylke]

Just a little update in my case. With cart with over 500 products in the page loads in about 25s when updated prestashop to latest 1.7.8.6. I have not found any dev who would know how to get any significant improvements. Im not a dev myself, but those doubled quotes for supplier are ridiculous since we don't even use suppliers 😅 @Hlavtox my prestashop wizard, maybe you faced similar issue among your clients?

[kpodemski]

@davidcylke I'm pretty sure at least 9k from this list is coming from the groupinc module. It depends on the manufacturers and suppliers

[thomas-villena]

Just for information, I followed step by step these recommandations and the performance was really better for me : https://devdocs.prestashop.com/1.7/scale/optimizations/

Hope this can help ..

[ChineseNorris]

3 years since pull was opened and no one is assigned yet to solve this issue... is this for real? This is a true real world scenario issue, with top priority (user can't finish order) and no one is assigned to solve this task yet.

I would really like to help but I don't know how to develop on Presta, where could I begin to learn to debug and develop?

PD: Would also like to pay if someone can fix this

[kandiaga]

Hi, Please conatct me from my website: https://prestatuts.com/en/contact-us I can help .

[chattago2002]

3 years since pull was opened and no one is assigned yet to solve this issue... is this for real? This is a true real world scenario issue, with top priority (user can't finish order) and no one is assigned to solve this task yet.

I would really like to help but I don't know how to develop on Presta, where could I begin to learn to debug and develop?

PD: Would also like to pay if someone can fix this

Really, I'm using PS from many years but sometimes seems to be abandoned to itself. I know PS is an open source project but to be credible and palatable it's difficult to accept more than 3 years with no a true answer nor a solution.

I have this problem with less then 20 products on the cart. For "cart update action" waiting time is about 2,50s and then "/ps_shoppingcart/ajax add-to-cart action" takes about other 1,50s.

[marsaldev]

Hi @chattago2002, I understand your frustration, and maybe also of others developers. But during this years developers (paid or from community) have worked on a lot of other stuff, for sure this is not an abandoned to itself project 😊

A lot of things have changed, and in the coming months there will be a person who will spend the 100% of time solving, or try to solve, performance issue(s). Again, paid by the company, so, 100% free for the community 😊

It is not an easy task, and we've already talked a lot internally about this topic (performance)

[davidcylke]

I have independent developer who fixed it for me. Lots of code to optimise, but now with 500 SKUs in cart it takes 2-3s to load, not 60 or more... If you work with devs from outside prestashop I can ask him if he is interested.

[mnowakpcgs]

It's most likely a module causing the slow load for you. The default install doesn't have this issue.

On Wed, Oct 26, 2022, 4:59 AM chattago2002 @.***> wrote:

3 years since pull was opened and no one is assigned yet to solve this issue... is this for real? This is a true real world scenario issue, with top priority (user can't finish order) and no one is assigned to solve this task yet.

I would really like to help but I don't know how to develop on Presta, where could I begin to learn to debug and develop?

PD: Would also like to pay if someone can fix this

Really, I'm using PS from many years but sometimes seems to be abandoned to itself. I know PS is an open source project but to be credible and palatable it's difficult to accept more than 3 years with no a true answer nor a solution.

I have this problem with less then 20 products on the cart. For "cart update action" waiting time is about 2,50s and then "/ps_shoppingcart/ajax add-to-cart action" takes about other 1,50s.

— Reply to this email directly, view it on GitHub https://github.com/PrestaShop/PrestaShop/issues/14979#issuecomment-1291857383, or unsubscribe https://github.com/notifications/unsubscribe-auth/APD3JZVMDY53MIOAM6ZVE2TWFEFKNANCNFSM4IJDO6KQ . You are receiving this because you were mentioned.Message ID: @.***>

[ChineseNorris]

I have independent developer who fixed it for me. Lots of code to optimise, but now with 500 SKUs in cart it takes 2-3s to load, not 60 or more... If you work with devs from outside prestashop I can ask him if he is interested.

would you mind to share those fixs to the community?

[davidcylke]

I would, but its not jest a simple file edit. I think he created cart cache or something to improve it. I can give you contact to him. I don't want to make any fake advertising here, but honestly I didn't find better independent devs through last 10 years.

[petyus-]

Hi @davidcylke , can you send me his contact in case he is interested on helping me too? ;)

[jon88seven]

Hi guys, Long story short, the problem is worse if you have features and combinations. The crazy thing is that the more your customers try to buy, the more you slow them down :) As soon as you added one product in cart, the code is so bad written that all the images, features, combinations, and all s**t related to that product are held. Add 30 products in cart and if you do not have a dedicated server you are out of business.

Also the prestashop demo https://demo.prestashop.com/#/en/front Add all the products and combination in cart (around 35) and you will notice that the popup confirmation is starting to be slow :) OMG. And there is no theme no module not even products, just think about it, their demo is beginning to be slow :)

Just view-source and look at " var prestashop = {"cart":{"products":..." , you'll se a huge pile of ...

How is its possible? Why they are not doing anything to prevent such thing to happen.

[davidcylke]

Yup, there is too much calculation and database queries each time you add something to cart. My programist solution is working fine, but I would rather like to use native implementation, instead of custom one which complicates updates and customization in the future.

[jon88seven]

Hi @davidcylke im frustrated because i migrated from 1.6 . Waited some years before i moved to 1.7 thinking that after so many releases the 1.7 is more stable and reliable. I think is my bad :) who would have thought.

I'm not a programmer but man this issue is huge. Why you let this issue to live so many years ??

The second most important thing is adding product to cart, the first is to checkout. Their entire function is to do this :) they are in the cart business.

@davidcylke can you share your programist solution / contact details? I don`t care about right now about updates and other customization. If the future is not sure. Thanks in advance

[davidcylke]

reach me by email djakdesign [at] gmail.com Im not sure if the progamist would like me to put his info here and probably its against the rules to advertise here.

[jon88seven]

@davidcylke thanks!

[kpodemski]

@davidcylke it's an open source software. If you had your developer improve the performance, you could ask him to submit his solution to the core so project members can review it, improve it, and implement it natively

[davidcylke]

@kpodemski I will ask him, but I think it was quite specific solution with some pricing caching and multiple workarounds, so it might not fit into presta's methods. But will talk to him.

[tonypartridge]

If he can work around it, it’s likely it can be incorporated into the core. In one way or another, a pull request into the project will allow plenty of others to see how it’s been done and different ways of improving it to be included into the core.

On 3 Nov 2022, at 08:55, davidcylke @.***> wrote:

@kpodemski https://github.com/kpodemski I will ask him, but I think it was quite specific solution with some pricing caching and multiple workarounds, so it might not fit into presta's methods. But will talk to him.

— Reply to this email directly, view it on GitHub https://github.com/PrestaShop/PrestaShop/issues/14979#issuecomment-1301801694, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKWBFXUOT5QGPTEXKE7H2TWGN4YDANCNFSM4IJDO6KQ. You are receiving this because you commented.

[kpodemski]

@tonypartridge exactly that :) this could be the first step to improving the current state of the performance inside the Cart

[davidcylke]

I know guys, but Im not able to demand from him to publish it.

[jon88seven]

@kpodemski yes, prestashop is free but everything else costs money. If something is open source its not a motive to think that is free, it has strings attached. You need server, modules, customizations, because nobody use it as it is (and they know it). They do business with it. They need money to do business. The basic product its free but for everything else you pay. I get it, its ok!

We are a community in symbiosis with PS Team But why they do not listen to our voice? at least with one ear! You think that they do not know how to solve this issue? Then its a problem!

If they know but do nothing then is not ok, its a big problem!

Allot of prestashop users are not skilled in programing we are just users and bug reporters. We contribute as we can. They are the creators they need to solve issues/bugs related to the product.

To be not able to sell products because your cart is getting slow, omg is hilarious, i cry and laugh in the same time!

[ocio87]

Hello, it still happens with version 1.7.8.7, there are more than 60 products in the cart, the slowness is generated by the ps_shoppingcart module

[mnowakpcgs]

Hello, it still happens with version 1.7.8.7, there are more than 60 products in the cart, the slowness is generated by the ps_shoppingcart module

Those queries have nothing to do with the shopping cart. It looks like a rewards module. If it's showing up under that module it's probably being done by ab override.

Even though there is a LIMIT 20, the HAVING clause is making it query the entire table and calculate the sum for each id_customer then returning only 20. I don't even see the purpose of that query. Why does a module displaying to individual customer need 20 random customers who have more than 5000 credits?

You can verify by disabling that module and any overrides its created, and your load time will probably go way down. If you feel that module is important, you need to fix these queries or work around them.

Something helpful to learn is append "EXPLAIN " at the beginning of a slow query and run it in phpmyadmin or mysql workbench. I've found horrendously slow queries before that was fixed by simply indexing a column, where it not being indexed was making mysql run through the entire table repeatedly for each individual row returned.

[davidcylke]

From what my developer told me it's caused by shopping cart and the fact that each page load all of the items it has are recalculated.

Not sure if it's just that.

In my case the problem was solved with a workaround, but I know lots of people still struggle with it and knowing this is not a top priority issue for prestashop developers I suggest creating a fundraiser. If someone knows how to do it technically without fear od scam please let us know.

Also, anyone from here would be willing to support it?

[mnowakpcgs]

Because if you disable all modules the problem does not exist. It's always some module causing it. Your developer should be able to identify the culprit.

The only way to solve that is better vetting and testing of modules. Half the time I buy a module it's more of a kit project than a functional module. Obviously this would drain a lot of resources to implement.

[Hlavtox]

The slowness comes from cart and cart rules. Your modules are only calling context->cart->getProducts(); or simmilar, which is so terribly slow. 👍

[thomas-villena]

Because if you disable all modules the problem does not exist. It's always some module causing it. Your developer should be able to identify the culprit.

The only way to solve that is better vetting and testing of modules. Half the time I buy a module it's more of a kit project than a functional module. Obviously this would drain a lot of resources to implement.

You forgot your previously written message in 21st of june 2020 ? You had respond to someone who prove that the problem exist with a new fresh prestashop installation without any additional plugin.

[mnowakpcgs]

I did not forget. I agreed that it was worth investigating. I was not the one claiming the issue exists on a clean install with no modules though.

I did follow that guidance and the cart was smooth on a clean install.

I also backported to 1.6 and with all modules the issue was still mitigated. So in my case it was indeed worse on 1.7.

Then since then I figured out why.

Lack of caching on shipping modules. I modified all shipping modules which utilize APIs to not only cache in DB but utilize the Cache class.

The reason it was worse on 1.7 is because of the split shipping feature. Rather then just querying the shipping APIs on every page load, it would query them for every single product in the shopping cart for every page load. 3 apis (fedex, ups, usps) for 30 products taking say 1-2s each query adds up to a massive wait time. Utilizing both DB cache and Cache class, thats 1-6s for the initial API queries and then subsequent loads are just 3 db queries and then senselessly but rapidly retrieving the same data from a php variable instead 90 times.

Their guidance is still correct that its most likely a module. The frustration is finding out which module and why.

I still also agree that the split shipping nonsense is more problematic than helpful. How many people are really placing orders online and asking different products to be shipped to different addresses? Who the hell requested that feature and who the hell thought its a brilliant idea?

[Hlavtox]

@mnowakpcgs Carrier module delays and API communication is not the point of this issue. The cause are terrible thousands of SQL queries because of cart rules and cart calculation. It's a core issue.

BTW - Multishipping to multiple addresses is nonsense, but it's important in case of multiple carriers. When product A can be sent by carrier A and product B with carrier B - multiple shipping.

Or, if people want to receive in stock items first.

[mnowakpcgs]

I've never seen cart rules causing that kind of degraded performance.

Do you have any specific functions identified that are causing this?

I'm currently on 1.6 and migrating to 8.0, but last I remember it didn't even do that. That would be logical to have a checkbox to split ship what is in stock and return a shipping cost for two separate shipments. I never used the feature but in the code it would request shipping cost for each item individually for it's respective address. Nobody ships like that, it would be obscenely expensive. And it doesn't even allow for the scenario you gave. Just shipping every item, in it's own box, to different addresses.

[ChineseNorris]

After doing some profiling in cart.php here is what I found out: getProducts() function is what does most of the slow queries where you can see because there are 11 JOINS in the same query, one order by and one sort by between joins... this is just okay for one product, but if you add more products to the query, especially products with combinations you end up with thousands of rows with only a few products in cart, not to mention all the duplicated rows done...

If you cut out the group by and sort by you can expect a 5% performance, if you cut out cells that you don't use (i.e. upc code, ecotax,manufacturers, etc) you can improve up to 10%, it's not much but still an improvement.

If anybody does know the fastest way to retrieve products in the cart would be nice, because how it's writter right now is quite painfull and hungry on resources

[davidcylke]

I'm willing to pay 1000 euro to anyone who will fix this as good as it could and and to make this as fast as it could be. I'm sure more people using prestashop for b2b would pay for it to be solved. Total could be worth the developers effort without workarounds.

Maybe this will make someone take this seriously after all those years ;) @Hlavtox maybe?:) I know you're good at improving prestashop.

[PR0SAT]

I reviewed my modules, disabled the not used one, having implemented cache and much free system resources, but the AJAX cart is really slow, I will have to do a migration to 8 ASAP, sadly the update proces gives me a f...k and shows Doctrine Schema erros which I'm unable to solve, gusess I will have to start with a clean install. You probably won't solve that issuse soon or never, right ? (no wonder with so many versions an the 8 in stable version)

[davidcylke]

You think someone reworked cart in Presta 8?

[PR0SAT]

You think someone reworked cart in Presta 8?

I have a clean 8 with imported products and it runs fast (AJAX CART) but the PHP ver. is also 8

[thomas-villena]

I can confirm that's not the case. Performance are still poor after migration.

[PR0SAT]

I suspect a JS timeout after response from server, there is somewhere a timeout dealy, because I get the response from the POST request, and the AJAX popup comes with increased delay aftwerwads

[thomas-villena]

The problem comes from the "present" function in Core which use an array_map several times to check all the last seen cart products even if you don't need it. This function recheck all products parameters.

[PR0SAT]

It could be, but also almost 3sec delay from JS script after receiving data

[PR0SAT]

The problem comes from the "present" function in Core which use an array_map several times to check all the last seen cart products even if you don't need it. This function recheck all products parameters.

Can you give us the the names of the PHP files? In my case I suspect the JS presta object, particulary the implementation of the ps_shoppingcart module.

[thomas-villena]

ps_shoppingcart module use the present function but it is not the only one. You will find it in : CartPresenter.php:392, PrestaShop\PrestaShop\Adapter\Presenter\Cart\CartPresenter->present()

[stefanovita]

We had a lot of problems with a B2B installation, where despite all the efforts to reduce loading speed, we had to develop a new cart controller module. In our installation, with the standard modules and theme, we had timeouts and 503 when more than 50 products were put in the cart. We had cart with more than 200 products and the website unresponsive. Thousands of queries on the standard controller. We developed a custom cart controller using datatables.net and we can manage cart of more than 300 products.

[kpodemski]

@stefanovita feel free to share your insights, so we could try to implement improvements into the core :)

[thomas-villena]

The main problem is that for each page, the cart is entirely reload several times due to the presence of _'cart' => $this->cart_presenter->present($this->context->cart),_ in the : protected function assignGeneralPurposeVariables()

It must be better to perform only the needed checks instead of global ones and to do it once. In the present method, the main lack of performance is the array_map which can take several seconds to execute. In our exemple, present is called 3 times for each catalog page.

[Hlavtox]

@thomas-villena This will be fixed by this PR - https://github.com/PrestaShop/PrestaShop/pull/31300, the results will be cached and returned immediately. :-)

I will test it and hopefully merge it today.

Then we can focus on improving the performance of the function. There is many things that is slow and hundreds of queries.

• Getting product data and price calculations.
• Getting carrier options and calculating it.
• Calculating and veryfing cart rules.

[stefanovita]

@stefanovita feel free to share your insights, so we could try to implement improvements into the core :)

Our version is a completely different controller, if you get in contact with me I can let you see how it works.