search

10up / ElasticPress

A fast and flexible search and query engine for WordPress.
https://elasticpress.io
GNU General Public License v2.0
1.24k stars 312 forks source link

Problem integrating ElasticPress and Dokan Multivendor #1937

Open gbvaz opened 3 years ago

gbvaz commented 3 years ago

Hey there!

I've been having trouble integrating ElasticPress and Dokan Multivendor plugins. This is not specific to the new v3.5, I've been having this problem since I first installed ElasticPress, which was the previous version, 3.4.3. I've opened a ticket to Dokan support as well, so they can help me out on their end, I'll keep you updated on their responses.

Dokan is a Marketplace plugin for Woocommerce, and it enables, among other things, the ability to sell user related products (Vendor role), splits commissions and so on. It has a dashboard, so vendors can manage their own products and orders.

We have almost 2k products by now and growing, so we needed a better search engine, that's when I came across ElasticPress (I'll call it EP from now on), which has been a great addition to our site, and I'm very grateful for that, thx!

I'm running my own ElasticSearch server, both locally for development and in live environments, under ubuntu servers in Digital Ocean.

The problem I've been facing is that when I have the Woocoomerce option enabled in EP's settings, Dokan's vendor's product dashboard won't display all products they have, it seems to be a problem filtering products with different status, also the can't add new products because the form redirects to a soft 404, when it should at least validate required fields, show validation messages or save a new product. With Woocommerce option disabled, everything works.

I've done some debugging, using both Query Monitor plugin (which is great because there's an option to set a cookie and have access to the debug tool logged in with a different user role other than admin, comes in handy with my multi vendor login) and EP recommendation, Debug Bar + 10up custom addition to EP debug. I was able to isolate a query that was once done in MySQL with EP disabled, and then moved to ElasticSearch query with EP enabled, that seems to be the problem with the product listing missing some products with different status.

This query:

SELECT SQL_CALC_FOUND_ROWS hAA_posts.ID
FROM hAA_posts
WHERE 1=1
AND hAA_posts.post_author IN (140)
AND hAA_posts.post_type = 'product'
AND ((hAA_posts.post_status = 'publish'
OR hAA_posts.post_status = 'future'
OR hAA_posts.post_status = 'draft'
OR hAA_posts.post_status = 'pending'))
GROUP BY hAA_posts.ID
ORDER BY hAA_posts.post_date DESC
LIMIT 0, 15

It returns 3 posts, being 2 published and 1 pending: 

id
20486
20481
20480

Is converted to this cURL request:

curl -XPOST -H 'Content-Type: application/json' -d '{"from":0,"size":15,"sort":[{"post_date":{"order":"desc"}}],"query":{"match_all":{"boost":1}},"post_filter":{"bool":{"must":[{"bool":{"must_not":[{"terms":{"terms.product_type.slug":[]}}]}},{"term":{"post_author.id":"140"}},{"terms":{"post_type.raw":["product"]}},{"terms":{"post_status":["publish","draft","pending","future"]}}]}}}' 'http://localhost:9200/localhostcityshoppe-post-1/_search'

It returns 2 posts, both published:

{
   "took":15,
   "timed_out":false,
   "_shards":{
      "total":5,
      "successful":5,
      "skipped":0,
      "failed":0
   },
   "hits":{
      "total":{
         "value":2,
         "relation":"eq"
      },
      "max_score":null,
      "hits":[
         {
            "_index":"localhostcityshoppe-post-1",
            "_type":"_doc",
            "_id":"20486",
            "_score":null,
            "_source":{
               "post_id":20486,
               "ID":20486,
               "post_author":{
                  "raw":"bsanborn",
                  "login":"bsanborn",
                  "id":140
               },
               "post_date":"2020-10-20 13:56:12",
               "post_date_gmt":"2020-10-20 20:56:12",
               "post_title":"test",
               "post_excerpt":"",
               "post_content_filtered":"",
               "post_content":"",
               "post_status":"publish",
               "post_name":"test",
               "post_modified":"2020-10-20 13:56:20",
               "post_modified_gmt":"2020-10-20 20:56:20",
               "post_parent":0,
               "post_type":"product",
               "post_mime_type":"",
               "permalink":"http:\/\/localhost\/cityshoppe\/product\/test\/",
               "terms":{
                  "product_cat":[
                     {
                        "term_id":2316,
                        "slug":"earrings",
                        "name":"Earrings",
                        "parent":25,
                        "term_taxonomy_id":2316,
                        "term_order":0
                     },
                     {
                        "term_id":25,
                        "slug":"jewelry",
                        "name":"Jewelry",
                        "parent":0,
                        "term_order":0
                     }
                  ],
                  "product_tag":[
                     {
                        "term_id":3994,
                        "slug":"guilherme",
                        "name":"guilherme",
                        "parent":0,
                        "term_taxonomy_id":3994,
                        "term_order":0
                     },
                     {
                        "term_id":3993,
                        "slug":"test",
                        "name":"test",
                        "parent":0,
                        "term_taxonomy_id":3993,
                        "term_order":0
                     }
                  ],
                  "product_type":[
                     {
                        "term_id":2,
                        "slug":"simple",
                        "name":"simple",
                        "parent":0,
                        "term_taxonomy_id":2,
                        "term_order":0
                     }
                  ]
               },
               "meta":{
                  "_regular_price":[
                     {
                        "value":"10",
                        "raw":"10",
                        "long":10,
                        "double":10,
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_sale_price":[
                     {
                        "value":"",
                        "raw":"",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_sale_price_dates_from":[
                     {
                        "value":"",
                        "raw":"",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_sale_price_dates_to":[
                     {
                        "value":"",
                        "raw":"",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_price":[
                     {
                        "value":"10",
                        "raw":"10",
                        "long":10,
                        "double":10,
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_visibility":[
                     {
                        "value":"visible",
                        "raw":"visible",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "total_sales":[
                     {
                        "value":"0",
                        "raw":"0",
                        "long":0,
                        "double":0,
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_tax_status":[
                     {
                        "value":"taxable",
                        "raw":"taxable",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_tax_class":[
                     {
                        "value":"",
                        "raw":"",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_manage_stock":[
                     {
                        "value":"no",
                        "raw":"no",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_backorders":[
                     {
                        "value":"no",
                        "raw":"no",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_sold_individually":[
                     {
                        "value":"no",
                        "raw":"no",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_virtual":[
                     {
                        "value":"no",
                        "raw":"no",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_downloadable":[
                     {
                        "value":"no",
                        "raw":"no",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_stock":[
                     {
                        "value":null,
                        "raw":null,
                        "boolean":false
                     }
                  ],
                  "_stock_status":[
                     {
                        "value":"instock",
                        "raw":"instock",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_wc_average_rating":[
                     {
                        "value":"0",
                        "raw":"0",
                        "long":0,
                        "double":0,
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_product_version":[
                     {
                        "value":"4.4.1",
                        "raw":"4.4.1",
                        "boolean":false,
                        "date":"2020-10-27",
                        "datetime":"2020-10-27 04:04:01",
                        "time":"04:04:01"
                     }
                  ],
                  "more_about_product_content":[
                     {
                        "value":"",
                        "raw":"",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "additional_details":[
                     {
                        "value":"",
                        "raw":"",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ]
               },
               "date_terms":{
                  "year":2020,
                  "month":10,
                  "week":43,
                  "dayofyear":293,
                  "day":20,
                  "dayofweek":2,
                  "dayofweek_iso":2,
                  "hour":13,
                  "minute":56,
                  "second":12,
                  "m":202010
               },
               "comment_count":0,
               "comment_status":"open",
               "ping_status":"closed",
               "menu_order":0,
               "guid":"http:\/\/localhost\/cityshoppe\/?post_type=product&p=20486"
            },
            "sort":[
               1603202172000
            ]
         },
         {
            "_index":"localhostcityshoppe-post-1",
            "_type":"_doc",
            "_id":"20481",
            "_score":null,
            "_source":{
               "post_id":20481,
               "ID":20481,
               "post_author":{
                  "raw":"bsanborn",
                  "login":"bsanborn",
                  "id":140
               },
               "post_date":"2020-10-15 18:54:58",
               "post_date_gmt":"2020-10-16 01:54:58",
               "post_title":"glass",
               "post_excerpt":"",
               "post_content_filtered":"",
               "post_content":"",
               "post_status":"publish",
               "post_name":"glass",
               "post_modified":"2020-10-15 18:55:09",
               "post_modified_gmt":"2020-10-16 01:55:09",
               "post_parent":0,
               "post_type":"product",
               "post_mime_type":"",
               "permalink":"http:\/\/localhost\/cityshoppe\/product\/glass\/",
               "terms":{
                  "product_cat":[
                     {
                        "term_id":3843,
                        "slug":"drop-earrings",
                        "name":"Drop Earrings",
                        "parent":2316,
                        "term_taxonomy_id":3843,
                        "term_order":0
                     },
                     {
                        "term_id":2316,
                        "slug":"earrings",
                        "name":"Earrings",
                        "parent":25,
                        "term_order":0
                     },
                     {
                        "term_id":25,
                        "slug":"jewelry",
                        "name":"Jewelry",
                        "parent":0,
                        "term_order":0
                     }
                  ],
                  "product_tag":[
                     {
                        "term_id":3991,
                        "slug":"asd",
                        "name":"asd",
                        "parent":0,
                        "term_taxonomy_id":3991,
                        "term_order":0
                     },
                     {
                        "term_id":3992,
                        "slug":"asdasd",
                        "name":"asdasd",
                        "parent":0,
                        "term_taxonomy_id":3992,
                        "term_order":0
                     },
                     {
                        "term_id":2282,
                        "slug":"glass",
                        "name":"glass",
                        "parent":0,
                        "term_taxonomy_id":2282,
                        "term_order":0
                     }
                  ],
                  "product_type":[
                     {
                        "term_id":2,
                        "slug":"simple",
                        "name":"simple",
                        "parent":0,
                        "term_taxonomy_id":2,
                        "term_order":0
                     }
                  ]
               },
               "meta":{
                  "_regular_price":[
                     {
                        "value":"19",
                        "raw":"19",
                        "long":19,
                        "double":19,
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_sale_price":[
                     {
                        "value":"",
                        "raw":"",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_sale_price_dates_from":[
                     {
                        "value":"",
                        "raw":"",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_sale_price_dates_to":[
                     {
                        "value":"",
                        "raw":"",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_price":[
                     {
                        "value":"19",
                        "raw":"19",
                        "long":19,
                        "double":19,
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_visibility":[
                     {
                        "value":"visible",
                        "raw":"visible",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "total_sales":[
                     {
                        "value":"0",
                        "raw":"0",
                        "long":0,
                        "double":0,
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_tax_status":[
                     {
                        "value":"taxable",
                        "raw":"taxable",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_tax_class":[
                     {
                        "value":"",
                        "raw":"",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_manage_stock":[
                     {
                        "value":"no",
                        "raw":"no",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_backorders":[
                     {
                        "value":"no",
                        "raw":"no",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_sold_individually":[
                     {
                        "value":"no",
                        "raw":"no",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_virtual":[
                     {
                        "value":"no",
                        "raw":"no",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_downloadable":[
                     {
                        "value":"no",
                        "raw":"no",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_stock":[
                     {
                        "value":null,
                        "raw":null,
                        "boolean":false
                     }
                  ],
                  "_stock_status":[
                     {
                        "value":"instock",
                        "raw":"instock",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_wc_average_rating":[
                     {
                        "value":"0",
                        "raw":"0",
                        "long":0,
                        "double":0,
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "_product_version":[
                     {
                        "value":"4.4.1",
                        "raw":"4.4.1",
                        "boolean":false,
                        "date":"2020-10-27",
                        "datetime":"2020-10-27 04:04:01",
                        "time":"04:04:01"
                     }
                  ],
                  "more_about_product_content":[
                     {
                        "value":"",
                        "raw":"",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "additional_details":[
                     {
                        "value":"",
                        "raw":"",
                        "boolean":false,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ],
                  "pageview":[
                     {
                        "value":"1",
                        "raw":"1",
                        "long":1,
                        "double":1,
                        "boolean":true,
                        "date":"1971-01-01",
                        "datetime":"1971-01-01 00:00:01",
                        "time":"00:00:01"
                     }
                  ]
               },
               "date_terms":{
                  "year":2020,
                  "month":10,
                  "week":42,
                  "dayofyear":288,
                  "day":15,
                  "dayofweek":4,
                  "dayofweek_iso":4,
                  "hour":18,
                  "minute":54,
                  "second":58,
                  "m":202010
               },
               "comment_count":0,
               "comment_status":"open",
               "ping_status":"closed",
               "menu_order":0,
               "guid":"http:\/\/localhost\/cityshoppe\/?post_type=product&p=20481"
            },
            "sort":[
               1602788098000
            ]
         }
      ]
   }
}

Product id 20480 is missing, which is the one with Pending status.

Now that's only half of the problem, as it only covers why pending products are not being shown in vendors dashboards. But I still need to figure out why I'm getting a soft 404 after I submit the new product form, but I'm still debugging and trying to figure it out. I'll get back to you when I have something.

Any help is appreciated! Thanks!

WP 5.5.1 Dokan Lite 3.0.11 (latest as of today) Dokan Pro 3.0.8 (latest as of today) ElasticPress 3.5 (latest as of today) Woocommerce 4.4.1

brandwaffle commented 3 years ago

@gbvaz by default EP does not index non-published content. You can activate Protected Content to index non-public content, or manually filter in the "Pending" status if you'd prefer. This is why EP isn't returning the pending post. To confirm, you can look at the outgoing request in the EP Debug Bar plugin (vs the returned results).

Be careful about indexing non-published content, as this could expose things like orders if your endpoint security isn't properly configured.

gbvaz commented 3 years ago

@brandwaffle thanks for the info. I've enabled Protected Content and it worked, now I can see the "pending" products in Dokan dashboard, which is great. I have a secure endpoint for the default indexing, but I'm not sure how can I secure it for the Autosuggest feature, as it does ajax requests, is there anywhere I can look into what needs to be done in order to have this kind of requests secure as well?

Anyways, I'm still having problems when I try to add a new product in Dokan dashboard using a vendor account. It throws me a 404 when it should validate the form and add a new product with "pending" status. I reached out to Dokan support but they claimed they couldn't help since they don't officially support ElasticPress. If I disable Woocommerce indexing in ElasticPress features, it works normally. It doesn't generate any logging errors, I'm lost here. What could be causing it?

oscarssanchez commented 3 years ago

Hi @gbvaz ,

Just to be sure we are on the same page, this problem happens with this plugin https://wordpress.org/plugins/dokan-lite/ and when you try to add a new product using a vendor account, correct?

gbvaz commented 3 years ago

Hi @oscarssanchez,

Yes, that's one. I've managed to go around the problem by editing Elasticpress in order to prevent it from querying elastic search in /dashboard pages in this file:

elasticpress/includes/classes/Feature/WooCommerce/WooCommerce.php

I've added this condition to the 'should_integrate_with_query' function:

if( isset($query->query['pagename']) && 'dashboard' === $query->query['pagename']){
    return false;
}

That's the page created by Dokan where we can add new products in the front-end. Obviously that's not a proper fix, as we don't get to use Elasticpress features in vendor's dashboard. Also, I did it like this because passing 'ep_integrate' => false, in Dokan's queries didn't work too. The proper fix would be making it work without any of those changes.