search

Ipstenu / varnish-http-purge

Proxy Cache Purge
Apache License 2.0
46 stars 48 forks source link

Deep Dive on what Woo does differently #63

Open gerdneuman opened 6 years ago

gerdneuman commented 6 years ago

Looking at the $purgeUrls param of the 'vhp_purge_urls' filter I see these URLs to be purged for a WooCommerce variable product:

https://www.mysite.de/de/shop/my-variable-product/?attribute_pa_weight=500g
https://www.mysite.de/de/shop/my-variable-product/?attribute_pa_weight=500gamp/
...
https://www.mysite.de/de/shop/my-variable-product/?attribute_pa_weight=500g/feed/
...

Not sure about the amp version (we are not using it), but the normal URLs should be this instead as far as I understand:

https://www.mysite.de/de/shop/my-variable-product/
...
https://www.mysite.de/de/shop/my-variable-product/feed/
...

That is the query parameter should omitted here.

Sure I could add the correct URLs myself using the 'vhp_purge_urls' filter but I think it's still a bug that not the correct URLs.

Ipstenu commented 6 years ago

Thank you for reporting this, I'll dig into and see if I can convince it to strip the params.

gerdneuman commented 6 years ago

@Ipstenu Thanks!

I also looked a bit into it and I think it might be due to woocommerce's get_permalink for variations:

https://github.com/woocommerce/woocommerce/blob/25be9fc13c67c9d2baf3139138a540756cb3f05f/includes/class-wc-product-variation.php#L151-L157

So maybe an approach would be something like this:

    $id = $postId;

    // otherwise use parent ID if variable product
    if ($postType == 'product_variation') {
        $variation = wc_get_product( $postId );
        $parentId = $variation->get_parent_id();
        $id = $parentId;
    }

Not sure it is enough to check for 'product_variation' which I got from https://docs.woocommerce.com/document/installed-taxonomies-post-types/#section-2

But there's also these grouped or external, but maybe they also use the 'product_variation' taxonomie:

https://github.com/woocommerce/woocommerce/blob/404fbca655c56436c94acfcf6fdf0aa22c0bdf7e/includes/wc-product-functions.php#L522-L537

I am not sure.

Also while you dig into it I came across a few questions while checking/testing the above:

  1. When doing a checkout with just 2 items of a certain variation (always the same 250 g weight variation) I noticed that vhp_purge_urls gets called like 6 times on checkout. Half of this is maybe due to Polylang calling it for the translated product, too, but still calling it 3 times with for the same $postid seems pretty much.

  2. Also, is the purge http calls actually blocking the checkout? (Checkout edit the products in the cart imho because the quantity and status like "sold out" / "available" is changed) I noticed that checkout takes a looong time (but can be other reasons as well). Of course it would be better if the Purging worked asynchroniously.

  3. I also noticed that the passed $urls array in vhp_purge_urls usually contains several duplicate items. Not sure if this means that there are also unnecessary http purge calls.

So, well, it would be great if you could look into

Ipstenu commented 6 years ago

I'm surprised that doing a checkout at all causes a purge unless that's updating a lot of page content. That is, if the checkout is triggering page saves with other pages, then that happens on the backend and would slow down a 'save' but I don't know WooCommerce that intimately :/

What duplicate items were you seeing in $urls? I can put in something to make sure all items are unique, which ... yeah, hi that would be smart!

My concern is what's the RIGHT way to flush those pages, since Varnish doesn't actually (or SHOULDN'T) cache anything with those params like ?whatever. So in theory the 'right' way would be:

1) Strip all query params from the urls 2) Ensure no URLs are duplicated in the flush call

gerdneuman commented 6 years ago

Hey, thanks for looking into it :)

I'm surprised that doing a checkout at all causes a purge unless that's updating a lot of page content. That is, if the checkout is triggering page saves with other pages, then that happens on the backend and would slow down a 'save' but I don't know WooCommerce that intimately :/

Oh, that's not so good, so it is blocking as far as I understand you...

I entered some loggin into our vhp_purge_urls filter. For each product (we mainly have variable products) the vhp_purge_urls filter is called 6 times on checkout. The checkout had two product (I search'n'replaced the real names):

[09-Aug-2018 19:30:27 UTC] LOGGING 01 | $postId: 5357 $purgeUrlsArray
(
    [0] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [1] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [2] => https://www.mysite.de/de/our-blog/author/myusername/
    [3] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [4] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [5] => 
    [6] => https://www.mysite.de/de/feed/rdf/
    [7] => https://www.mysite.de/de/feed/rss/
    [8] => https://www.mysite.de/de/feed/
    [9] => https://www.mysite.de/de/feed/atom/
    [10] => https://www.mysite.de/de/comments/feed/
    [11] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [12] => https://www.mysite.de/wp-json/
    [13] => https://www.mysite.de/
    [14] => https://www.mysite.de/de/our-blog/
)

[09-Aug-2018 19:30:27 UTC] LOGGING 01 | $postId: 5357 $purgeUrlsArray
(
    [0] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [1] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [2] => https://www.mysite.de/de/our-blog/author/myusername/
    [3] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [4] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [5] => 
    [6] => https://www.mysite.de/de/feed/rdf/
    [7] => https://www.mysite.de/de/feed/rss/
    [8] => https://www.mysite.de/de/feed/
    [9] => https://www.mysite.de/de/feed/atom/
    [10] => https://www.mysite.de/de/comments/feed/
    [11] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [12] => https://www.mysite.de/wp-json/
    [13] => https://www.mysite.de/
    [14] => https://www.mysite.de/de/our-blog/
    [15] => https://www.mysite.de/sitemap.xml
    [16] => https://www.mysite.de/de/shop/the-one-variable-product/
    [17] => https://www.mysite.de/en/shop/the-one-variable-product/
    [18] => https://www.mysite.de/de/shop/
    [19] => https://www.mysite.de/en/shop/
    [20] => https://www.mysite.de/sitemap-posttype-product.xml
    [21] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [22] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [23] => https://www.mysite.de/de/our-blog/author/myusername/
    [24] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [25] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [26] => 
    [27] => https://www.mysite.de/de/feed/rdf/
    [28] => https://www.mysite.de/de/feed/rss/
    [29] => https://www.mysite.de/de/feed/
    [30] => https://www.mysite.de/de/feed/atom/
    [31] => https://www.mysite.de/de/comments/feed/
    [32] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [33] => https://www.mysite.de/wp-json/
    [34] => https://www.mysite.de/
    [35] => https://www.mysite.de/de/our-blog/
)

[09-Aug-2018 19:30:27 UTC] LOGGING 01 | $postId: 8802 $purgeUrlsArray
(
    [0] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [1] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [2] => https://www.mysite.de/de/our-blog/author/myusername/
    [3] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [4] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [5] => 
    [6] => https://www.mysite.de/de/feed/rdf/
    [7] => https://www.mysite.de/de/feed/rss/
    [8] => https://www.mysite.de/de/feed/
    [9] => https://www.mysite.de/de/feed/atom/
    [10] => https://www.mysite.de/de/comments/feed/
    [11] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [12] => https://www.mysite.de/wp-json/
    [13] => https://www.mysite.de/
    [14] => https://www.mysite.de/de/our-blog/
    [15] => https://www.mysite.de/sitemap.xml
    [16] => https://www.mysite.de/de/shop/the-one-variable-product/
    [17] => https://www.mysite.de/en/shop/the-one-variable-product/
    [18] => https://www.mysite.de/de/shop/
    [19] => https://www.mysite.de/en/shop/
    [20] => https://www.mysite.de/sitemap-posttype-product.xml
    [42] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [43] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [44] => https://www.mysite.de/de/our-blog/author/myusername/
    [45] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [46] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [47] => 
    [48] => https://www.mysite.de/de/feed/rdf/
    [49] => https://www.mysite.de/de/feed/rss/
    [50] => https://www.mysite.de/de/feed/
    [51] => https://www.mysite.de/de/feed/atom/
    [52] => https://www.mysite.de/de/comments/feed/
    [53] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [54] => https://www.mysite.de/wp-json/
    [55] => https://www.mysite.de/
    [56] => https://www.mysite.de/de/our-blog/
)

[09-Aug-2018 19:30:27 UTC] LOGGING 01 | $postId: 8802 $purgeUrlsArray
(
    [0] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [1] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [2] => https://www.mysite.de/de/our-blog/author/myusername/
    [3] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [4] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [5] => 
    [6] => https://www.mysite.de/de/feed/rdf/
    [7] => https://www.mysite.de/de/feed/rss/
    [8] => https://www.mysite.de/de/feed/
    [9] => https://www.mysite.de/de/feed/atom/
    [10] => https://www.mysite.de/de/comments/feed/
    [11] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [12] => https://www.mysite.de/wp-json/
    [13] => https://www.mysite.de/
    [14] => https://www.mysite.de/de/our-blog/
    [15] => https://www.mysite.de/sitemap.xml
    [16] => https://www.mysite.de/de/shop/the-one-variable-product/
    [17] => https://www.mysite.de/en/shop/the-one-variable-product/
    [18] => https://www.mysite.de/de/shop/
    [19] => https://www.mysite.de/en/shop/
    [20] => https://www.mysite.de/sitemap-posttype-product.xml
    [21] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [22] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [32] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [42] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [43] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [44] => https://www.mysite.de/de/our-blog/author/myusername/
    [45] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [46] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [47] => 
    [48] => https://www.mysite.de/de/feed/rdf/
    [49] => https://www.mysite.de/de/feed/rss/
    [50] => https://www.mysite.de/de/feed/
    [51] => https://www.mysite.de/de/feed/atom/
    [52] => https://www.mysite.de/de/comments/feed/
    [53] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [54] => https://www.mysite.de/wp-json/
    [55] => https://www.mysite.de/
    [56] => https://www.mysite.de/de/our-blog/
)

[09-Aug-2018 19:30:27 UTC] LOGGING 01 | $postId: 5357 $purgeUrlsArray
(
    [0] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [1] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [2] => https://www.mysite.de/de/our-blog/author/myusername/
    [3] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [4] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [5] => 
    [6] => https://www.mysite.de/de/feed/rdf/
    [7] => https://www.mysite.de/de/feed/rss/
    [8] => https://www.mysite.de/de/feed/
    [9] => https://www.mysite.de/de/feed/atom/
    [10] => https://www.mysite.de/de/comments/feed/
    [11] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [12] => https://www.mysite.de/wp-json/
    [13] => https://www.mysite.de/
    [14] => https://www.mysite.de/de/our-blog/
    [15] => https://www.mysite.de/sitemap.xml
    [16] => https://www.mysite.de/de/shop/the-one-variable-product/
    [17] => https://www.mysite.de/en/shop/the-one-variable-product/
    [18] => https://www.mysite.de/de/shop/
    [19] => https://www.mysite.de/en/shop/
    [20] => https://www.mysite.de/sitemap-posttype-product.xml
    [21] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [22] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [23] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [45] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [46] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [47] => https://www.mysite.de/de/our-blog/author/myusername/
    [48] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [49] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [50] => 
    [51] => https://www.mysite.de/de/feed/rdf/
    [52] => https://www.mysite.de/de/feed/rss/
    [53] => https://www.mysite.de/de/feed/
    [54] => https://www.mysite.de/de/feed/atom/
    [55] => https://www.mysite.de/de/comments/feed/
    [56] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [57] => https://www.mysite.de/wp-json/
    [58] => https://www.mysite.de/
    [59] => https://www.mysite.de/de/our-blog/
)

[09-Aug-2018 19:30:27 UTC] LOGGING 01 | $postId: 5357 $purgeUrlsArray
(
    [0] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [1] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [2] => https://www.mysite.de/de/our-blog/author/myusername/
    [3] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [4] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [5] => 
    [6] => https://www.mysite.de/de/feed/rdf/
    [7] => https://www.mysite.de/de/feed/rss/
    [8] => https://www.mysite.de/de/feed/
    [9] => https://www.mysite.de/de/feed/atom/
    [10] => https://www.mysite.de/de/comments/feed/
    [11] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [12] => https://www.mysite.de/wp-json/
    [13] => https://www.mysite.de/
    [14] => https://www.mysite.de/de/our-blog/
    [15] => https://www.mysite.de/sitemap.xml
    [16] => https://www.mysite.de/de/shop/the-one-variable-product/
    [17] => https://www.mysite.de/en/shop/the-one-variable-product/
    [18] => https://www.mysite.de/de/shop/
    [19] => https://www.mysite.de/en/shop/
    [20] => https://www.mysite.de/sitemap-posttype-product.xml
    [21] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [22] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [23] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [45] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [46] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [47] => https://www.mysite.de/de/our-blog/author/myusername/
    [48] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [49] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [50] => 
    [51] => https://www.mysite.de/de/feed/rdf/
    [52] => https://www.mysite.de/de/feed/rss/
    [53] => https://www.mysite.de/de/feed/
    [54] => https://www.mysite.de/de/feed/atom/
    [55] => https://www.mysite.de/de/comments/feed/
    [56] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [57] => https://www.mysite.de/wp-json/
    [58] => https://www.mysite.de/
    [59] => https://www.mysite.de/de/our-blog/
)
[09-Aug-2018 19:30:27 UTC] LOGGING 01 | $postId: 4489 $purgeUrlsArray
(
    [0] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [1] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [2] => https://www.mysite.de/de/our-blog/author/myusername/
    [3] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [4] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [5] => 
    [6] => https://www.mysite.de/de/feed/rdf/
    [7] => https://www.mysite.de/de/feed/rss/
    [8] => https://www.mysite.de/de/feed/
    [9] => https://www.mysite.de/de/feed/atom/
    [10] => https://www.mysite.de/de/comments/feed/
    [11] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [12] => https://www.mysite.de/wp-json/
    [13] => https://www.mysite.de/
    [14] => https://www.mysite.de/de/our-blog/
    [15] => https://www.mysite.de/sitemap.xml
    [16] => https://www.mysite.de/de/shop/the-one-variable-product/
    [17] => https://www.mysite.de/en/shop/the-one-variable-product/
    [18] => https://www.mysite.de/de/shop/
    [19] => https://www.mysite.de/en/shop/
    [20] => https://www.mysite.de/sitemap-posttype-product.xml
    [21] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [22] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [23] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [45] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500g
    [46] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500gamp/
    [47] => https://www.mysite.de/de/our-blog/author/myusername/
    [48] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [49] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [50] => 
    [51] => https://www.mysite.de/de/feed/rdf/
    [52] => https://www.mysite.de/de/feed/rss/
    [53] => https://www.mysite.de/de/feed/
    [54] => https://www.mysite.de/de/feed/atom/
    [55] => https://www.mysite.de/de/comments/feed/
    [56] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500g/feed/
    [57] => https://www.mysite.de/wp-json/
    [58] => https://www.mysite.de/
    [59] => https://www.mysite.de/de/our-blog/
)

[09-Aug-2018 19:30:27 UTC] LOGGING 01 | $postId: 4489 $purgeUrlsArray
(
    [0] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [1] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [2] => https://www.mysite.de/de/our-blog/author/myusername/
    [3] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [4] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [5] => 
    [6] => https://www.mysite.de/de/feed/rdf/
    [7] => https://www.mysite.de/de/feed/rss/
    [8] => https://www.mysite.de/de/feed/
    [9] => https://www.mysite.de/de/feed/atom/
    [10] => https://www.mysite.de/de/comments/feed/
    [11] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [12] => https://www.mysite.de/wp-json/
    [13] => https://www.mysite.de/
    [14] => https://www.mysite.de/de/our-blog/
    [15] => https://www.mysite.de/sitemap.xml
    [16] => https://www.mysite.de/de/shop/the-one-variable-product/
    [17] => https://www.mysite.de/en/shop/the-one-variable-product/
    [18] => https://www.mysite.de/de/shop/
    [19] => https://www.mysite.de/en/shop/
    [20] => https://www.mysite.de/sitemap-posttype-product.xml
    [21] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [22] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [23] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [24] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500g
    [25] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500gamp/
    [35] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500g/feed/
    [40] => https://www.mysite.de/de/shop/some-other-variable-product/
    [41] => https://www.mysite.de/en/shop/blackcap-espresso/
    [45] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500g
    [46] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500gamp/
    [47] => https://www.mysite.de/de/our-blog/author/myusername/
    [48] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [49] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [50] => 
    [51] => https://www.mysite.de/de/feed/rdf/
    [52] => https://www.mysite.de/de/feed/rss/
    [53] => https://www.mysite.de/de/feed/
    [54] => https://www.mysite.de/de/feed/atom/
    [55] => https://www.mysite.de/de/comments/feed/
    [56] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500g/feed/
    [57] => https://www.mysite.de/wp-json/
    [58] => https://www.mysite.de/
    [59] => https://www.mysite.de/de/our-blog/
)
[09-Aug-2018 19:30:27 UTC] LOGGING 01 | $postId: 7759 $purgeUrlsArray
(
    [0] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [1] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [2] => https://www.mysite.de/de/our-blog/author/myusername/
    [3] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [4] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [5] => 
    [6] => https://www.mysite.de/de/feed/rdf/
    [7] => https://www.mysite.de/de/feed/rss/
    [8] => https://www.mysite.de/de/feed/
    [9] => https://www.mysite.de/de/feed/atom/
    [10] => https://www.mysite.de/de/comments/feed/
    [11] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [12] => https://www.mysite.de/wp-json/
    [13] => https://www.mysite.de/
    [14] => https://www.mysite.de/de/our-blog/
    [15] => https://www.mysite.de/sitemap.xml
    [16] => https://www.mysite.de/de/shop/the-one-variable-product/
    [17] => https://www.mysite.de/en/shop/the-one-variable-product/
    [18] => https://www.mysite.de/de/shop/
    [19] => https://www.mysite.de/en/shop/
    [20] => https://www.mysite.de/sitemap-posttype-product.xml
    [21] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [22] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [23] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [24] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500g
    [25] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500gamp/
    [26] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500g/feed/
    [27] => https://www.mysite.de/de/shop/some-other-variable-product/
    [28] => https://www.mysite.de/en/shop/blackcap-espresso/
    [50] => https://www.mysite.de/en/shop/blackcap-espresso/?attribute_pa_weight=500g
    [51] => https://www.mysite.de/en/shop/blackcap-espresso/?attribute_pa_weight=500gamp/
    [52] => https://www.mysite.de/de/our-blog/author/myusername/
    [53] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [54] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [55] => 
    [56] => https://www.mysite.de/de/feed/rdf/
    [57] => https://www.mysite.de/de/feed/rss/
    [58] => https://www.mysite.de/de/feed/
    [59] => https://www.mysite.de/de/feed/atom/
    [60] => https://www.mysite.de/de/comments/feed/
    [61] => https://www.mysite.de/en/shop/blackcap-espresso/?attribute_pa_weight=500g/feed/
    [62] => https://www.mysite.de/wp-json/
    [63] => https://www.mysite.de/
    [64] => https://www.mysite.de/de/our-blog/
)
[09-Aug-2018 19:30:27 UTC] LOGGING 01 | $postId: 7759 $purgeUrlsArray
(
    [0] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [1] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [2] => https://www.mysite.de/de/our-blog/author/myusername/
    [3] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [4] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [5] => 
    [6] => https://www.mysite.de/de/feed/rdf/
    [7] => https://www.mysite.de/de/feed/rss/
    [8] => https://www.mysite.de/de/feed/
    [9] => https://www.mysite.de/de/feed/atom/
    [10] => https://www.mysite.de/de/comments/feed/
    [11] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [12] => https://www.mysite.de/wp-json/
    [13] => https://www.mysite.de/
    [14] => https://www.mysite.de/de/our-blog/
    [15] => https://www.mysite.de/sitemap.xml
    [16] => https://www.mysite.de/de/shop/the-one-variable-product/
    [17] => https://www.mysite.de/en/shop/the-one-variable-product/
    [18] => https://www.mysite.de/de/shop/
    [19] => https://www.mysite.de/en/shop/
    [20] => https://www.mysite.de/sitemap-posttype-product.xml
    [21] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [22] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [23] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [24] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500g
    [25] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500gamp/
    [26] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500g/feed/
    [27] => https://www.mysite.de/de/shop/some-other-variable-product/
    [28] => https://www.mysite.de/en/shop/blackcap-espresso/
    [29] => https://www.mysite.de/en/shop/blackcap-espresso/?attribute_pa_weight=500g
    [30] => https://www.mysite.de/en/shop/blackcap-espresso/?attribute_pa_weight=500gamp/
    [40] => https://www.mysite.de/en/shop/blackcap-espresso/?attribute_pa_weight=500g/feed/
    [50] => https://www.mysite.de/en/shop/blackcap-espresso/?attribute_pa_weight=500g
    [51] => https://www.mysite.de/en/shop/blackcap-espresso/?attribute_pa_weight=500gamp/
    [52] => https://www.mysite.de/de/our-blog/author/myusername/
    [53] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [54] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [55] => 
    [56] => https://www.mysite.de/de/feed/rdf/
    [57] => https://www.mysite.de/de/feed/rss/
    [58] => https://www.mysite.de/de/feed/
    [59] => https://www.mysite.de/de/feed/atom/
    [60] => https://www.mysite.de/de/comments/feed/
    [61] => https://www.mysite.de/en/shop/blackcap-espresso/?attribute_pa_weight=500g/feed/
    [62] => https://www.mysite.de/wp-json/
    [63] => https://www.mysite.de/
    [64] => https://www.mysite.de/de/our-blog/
)
[09-Aug-2018 19:30:27 UTC] LOGGING 01 | $postId: 4489 $purgeUrlsArray
(
    [0] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [1] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [2] => https://www.mysite.de/de/our-blog/author/myusername/
    [3] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [4] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [5] => 
    [6] => https://www.mysite.de/de/feed/rdf/
    [7] => https://www.mysite.de/de/feed/rss/
    [8] => https://www.mysite.de/de/feed/
    [9] => https://www.mysite.de/de/feed/atom/
    [10] => https://www.mysite.de/de/comments/feed/
    [11] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [12] => https://www.mysite.de/wp-json/
    [13] => https://www.mysite.de/
    [14] => https://www.mysite.de/de/our-blog/
    [15] => https://www.mysite.de/sitemap.xml
    [16] => https://www.mysite.de/de/shop/the-one-variable-product/
    [17] => https://www.mysite.de/en/shop/the-one-variable-product/
    [18] => https://www.mysite.de/de/shop/
    [19] => https://www.mysite.de/en/shop/
    [20] => https://www.mysite.de/sitemap-posttype-product.xml
    [21] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [22] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [23] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [24] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500g
    [25] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500gamp/
    [26] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500g/feed/
    [27] => https://www.mysite.de/de/shop/some-other-variable-product/
    [28] => https://www.mysite.de/en/shop/blackcap-espresso/
    [29] => https://www.mysite.de/en/shop/blackcap-espresso/?attribute_pa_weight=500g
    [30] => https://www.mysite.de/en/shop/blackcap-espresso/?attribute_pa_weight=500gamp/
    [31] => https://www.mysite.de/en/shop/blackcap-espresso/?attribute_pa_weight=500g/feed/
    [53] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500g
    [54] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500gamp/
    [55] => https://www.mysite.de/de/our-blog/author/myusername/
    [56] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [57] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [58] => 
    [59] => https://www.mysite.de/de/feed/rdf/
    [60] => https://www.mysite.de/de/feed/rss/
    [61] => https://www.mysite.de/de/feed/
    [62] => https://www.mysite.de/de/feed/atom/
    [63] => https://www.mysite.de/de/comments/feed/
    [64] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500g/feed/
    [65] => https://www.mysite.de/wp-json/
    [66] => https://www.mysite.de/
    [67] => https://www.mysite.de/de/our-blog/
)
[09-Aug-2018 19:30:27 UTC] LOGGING 01 | $postId: 4489 $purgeUrlsArray
(
    [0] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [1] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [2] => https://www.mysite.de/de/our-blog/author/myusername/
    [3] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [4] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [5] => 
    [6] => https://www.mysite.de/de/feed/rdf/
    [7] => https://www.mysite.de/de/feed/rss/
    [8] => https://www.mysite.de/de/feed/
    [9] => https://www.mysite.de/de/feed/atom/
    [10] => https://www.mysite.de/de/comments/feed/
    [11] => https://www.mysite.de/de/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [12] => https://www.mysite.de/wp-json/
    [13] => https://www.mysite.de/
    [14] => https://www.mysite.de/de/our-blog/
    [15] => https://www.mysite.de/sitemap.xml
    [16] => https://www.mysite.de/de/shop/the-one-variable-product/
    [17] => https://www.mysite.de/en/shop/the-one-variable-product/
    [18] => https://www.mysite.de/de/shop/
    [19] => https://www.mysite.de/en/shop/
    [20] => https://www.mysite.de/sitemap-posttype-product.xml
    [21] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg
    [22] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kgamp/
    [23] => https://www.mysite.de/en/shop/the-one-variable-product/?attribute_pa_weight=1kg/feed/
    [24] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500g
    [25] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500gamp/
    [26] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500g/feed/
    [27] => https://www.mysite.de/de/shop/some-other-variable-product/
    [28] => https://www.mysite.de/en/shop/blackcap-espresso/
    [29] => https://www.mysite.de/en/shop/blackcap-espresso/?attribute_pa_weight=500g
    [30] => https://www.mysite.de/en/shop/blackcap-espresso/?attribute_pa_weight=500gamp/
    [31] => https://www.mysite.de/en/shop/blackcap-espresso/?attribute_pa_weight=500g/feed/
    [53] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500g
    [54] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500gamp/
    [55] => https://www.mysite.de/de/our-blog/author/myusername/
    [56] => https://www.mysite.de/de/our-blog/author/myusername/feed/
    [57] => https://www.mysite.de/wp-json/wp/v2/users/5/
    [58] => 
    [59] => https://www.mysite.de/de/feed/rdf/
    [60] => https://www.mysite.de/de/feed/rss/
    [61] => https://www.mysite.de/de/feed/
    [62] => https://www.mysite.de/de/feed/atom/
    [63] => https://www.mysite.de/de/comments/feed/
    [64] => https://www.mysite.de/de/shop/some-other-variable-product/?attribute_pa_weight=500g/feed/
    [65] => https://www.mysite.de/wp-json/
    [66] => https://www.mysite.de/
    [67] => https://www.mysite.de/de/our-blog/
)

As you can see the $urls array as passed to the filter gets longer and longer (piles up somehow.) Hence the duplicate URLs.

  1. Strip all query params from the urls

Yeah, you could either strip everything including and after the ?. Or use the code I posted at https://github.com/Ipstenu/varnish-http-purge/issues/63#issuecomment-411815576 to get the parent product of the variation product. In the end it had the same effect. Stripping the ? might be a more general solution.

OTOH, I've seen calls to vhp_purge_urls like the following on a daily basis:

[10-Aug-2018 05:48:54 UTC] LOGGING 01 | $postId: 15852 $purgeUrlsArray
(
    [0] => https://www.mysite.de/?post_type=scheduled-action&p=15852
    [1] => https://www.mysite.de/?post_type=scheduled-action&p=15852amp/
... same as above...
    [11] => https://www.mysite.de/?post_type=scheduled-action&p=15852/feed/
...
)

Not sure what this is. I could not find a 15852 in our system, but it also includes a ? param string.

As far the blocking, I think there are ways to run a command asynchronously in WordPress. Our orders usually contain about 4 products on average, sometimes about 8. That would be 4 x 6 = 24 or 8 x 6 = 48 purge calls on checkout at the moment, just to give some numbers... Quite a bummer :-(

At the moment we're discussion 3 different issues: query params for product variations, duplicate URLs and blocking on checkout. (Sorry, my fault that I mixed this all in here). Maybe the last two should be put into different issue to be addressed separately... Not sure how you'd like to handle it?

gerdneuman commented 6 years ago

Regarding blocking: Just googled a bit and came up with these two things that look promising:

Ipstenu commented 6 years ago

p=15852 is your post ID. You have 15852 post ids.

Okay, for sanity since I'm about to turn into Road Warrior Ipstenu in a couple days:

I'm leaving this open and about to rename it to "Understanding Woo!"

gerdneuman commented 6 years ago

I've added another issue #69 (to make this meta ticket more split up in approachable issues) which is the biggest bummer at the moment when using varnish-http-purge together with WooCommerce since it makes the checkout http request 30s-60s slower than it should be.