meilisearch / strapi-plugin-meilisearch

A strapi plugin to add your collections to Meilisearch
https://meilisearch.com
MIT License
223 stars 58 forks source link

Not able to populate all data by Meilisearch pluging using populateEntryRule in my strapi project #506

Closed oluademola closed 2 years ago

oluademola commented 2 years ago

Discussed in https://github.com/meilisearch/meilisearch/discussions/2747

Originally posted by **giri943** September 6, 2022 I am trying to integrate meilisearch in my Strapi Project. While integrating and testing meilisearch I found that the plugin is failing to fetch all the data, especially when the data is located inside a deeply nested array. The plugin successfully fetches the data from the first level of nested array, rest of the datas are not being fetched by meilisearch plugin. I have tried using the populateEntryRule on plugin.js file which I found from the meilisearch documentation but still it's not fetching all the data. When I fire an API call to that particular collection type through Strapi(not through meilisearch) am getting all the data correctly. Here is my particular schema that I am working with: ``` { "kind": "collectionType", "collectionName": "investors", "info": { "singularName": "investor", "pluralName": "investors", "displayName": "Investor", "description": "" }, "options": { "draftAndPublish": true }, "pluginOptions": {}, "attributes": { "title": { "type": "string" }, "slug": { "type": "uid", "targetField": "title" }, "metaData": { "type": "component", "repeatable": false, "component": "seo.meta-fields" }, "pageSections": { "type": "dynamiczone", "components": [ "sections.hero", "sections.certification-section", "sections.community-section", "sections.content-section", "sections.home-page-section-2", "sections.lm-business-modal-section", "sections.lm-evolving-section", "sections.lm-leardership-section", "sections.milestone-section", "sections.mission-section", "sections.our-business-section", "sections.product-hero-section", "sections.statistics-section", "sections.team-section", "sections.value-section", "sections.vision-section", "sections.webcast-section", "sections.chart-section", "sections.key-financials", "sections.fixed-deposit", "sections.financial-performance" ] }, "Accordion": { "type": "dynamiczone", "components": [ "elements.no-dropdown", "elements.dropdown" ] } } } ``` I am having trouble in fetching the data from "pageSection" which has some deeply nested arrays of data. My Meilisearch populateEntryRule plugins.js file ``` module.exports = { meilisearch: { config: { investor: { populateEntryRule:['dynamiczone.pageSections', 'pageSections'] } } }, } ``` Strapi Content type structure: ![StrapiContent Structure](https://user-images.githubusercontent.com/74488003/188640322-a2fa1df9-790a-4f17-9d17-53626ab6ff6b.jpg) Strapi API call to a collection ![api call](https://user-images.githubusercontent.com/74488003/188639999-324f50bd-330c-4261-87f4-351616bd7c86.jpg) Meilisearch Result ![FrontEnd Data](https://user-images.githubusercontent.com/74488003/188640116-caa77247-73f1-413d-8f7f-6f2726c596ab.jpg) I couldn't find any related articles anywhere, that's why I opened a discussion over here. Kindlylook into my problem and any help or suggestion will be highly appreciated.
bidoubiwa commented 2 years ago

Hello @oluademola

This behavior is very weird as we do not alter the populate rule before sending it to the Strapi API.

https://github.com/meilisearch/strapi-plugin-meilisearch/blob/e7e65609187c01fcbb6f86f7982d6b0245af8442/server/services/content-types/content-types.js#L167-L193

When I fire an API call to that particular collection type through Strapi(not through meilisearch) am getting all the data correctly.

Are you using the same Strapi API as the one from the code example above?

hamzaahmed074 commented 2 years ago

I am facing the same problem @bidoubiwa my case is I have to tables Inventory and product inventory has one product there can be multiple inventory that targets a single product so common items/fields are in product. When i index inventory i get all the fields in product but not products media items/ images or any other relationship product has This is my Inventory { "kind": "collectionType", "collectionName": "inventories", "info": { "singularName": "inventory", "pluralName": "inventories", "displayName": "Inventory", "description": "" }, "options": { "draftAndPublish": true }, "pluginOptions": {}, "attributes": { "retailer": { "type": "relation", "relation": "oneToOne", "target": "api::retailer.retailer" }, "product": { "type": "relation", "relation": "oneToOne", "target": "api::product.product" }, "productERPId": { "type": "string", "unique": true }, "quantity": { "type": "integer", "min": 0, "default": 0, "required": true }, "priceMed": { "type": "decimal", "min": 0 }, "priceRec": { "type": "decimal", "min": 0 }, "specialPriceMed": { "type": "decimal", "min": 0 }, "specialPriceRec": { "type": "decimal", "min": 0, "required": false, "default": 0 }, "ERPSyncedAt": { "type": "datetime" }, "ordering": { "type": "integer", "default": 1000000, "required": false, "min": 0 }, "weight": { "type": "string", "required": true }, "thcMg": { "type": "decimal", "required": true, "min": 0, "default": 0 }, "cbdFormatted": { "type": "string" }, "thcFormatted": { "type": "string" }, "inventoryMenuType": { "type": "enumeration", "enum": [ "MEDICAL", "RECREATIONAL", "BOTH" ] }, "activeTerpenes": { "type": "component", "repeatable": true, "component": "product.active-terpene" }, "seo": { "type": "component", "repeatable": false, "component": "shared.seo" }, "specials": { "type": "relation", "relation": "manyToMany", "target": "api::special.special", "mappedBy": "inventories" } } } This is my Product { "kind": "collectionType", "collectionName": "products", "info": { "singularName": "product", "pluralName": "products", "displayName": "Product", "description": "" }, "options": { "draftAndPublish": true }, "pluginOptions": {}, "attributes": { "nameCopy": { "type": "string" }, "ERPId": { "type": "string", "required": true, "unique": true }, "slug": { "type": "uid", "targetField": "nameCopy", "required": true }, "description": { "type": "text" }, "onset": { "type": "string" }, "duration": { "type": "string" }, "relaxed": { "type": "integer", "max": 100, "min": 0 }, "happy": { "type": "integer", "max": 100, "min": 0 }, "focused": { "type": "integer", "max": 100, "min": 0 }, "hungry": { "type": "integer", "max": 100, "min": 0 }, "sleepy": { "type": "integer", "max": 100, "min": 0 }, "painRelief": { "type": "integer", "max": 100, "min": 0 }, "terpenes": { "type": "relation", "relation": "oneToMany", "target": "api::terpene.terpene" }, "image": { "type": "media", "multiple": false, "required": false, "allowedTypes": [ "images" ] }, "ordering": { "type": "integer", "required": true, "min": 0, "default": 1000000 }, "coaUrl": { "type": "string", "regex": "(http|ftp|https)://[\\w-]+(\\.[\\w-]+)*([\\w.,@?^=%&amp;:/~+#-]*[\\w@?^=%&amp;/~+#-])?" }, "brand": { "type": "relation", "relation": "oneToOne", "target": "api::brand.brand" }, "category": { "type": "relation", "relation": "oneToOne", "target": "api::category.category" }, "subcategory": { "type": "relation", "relation": "oneToOne", "target": "api::subcategory.subcategory" }, "strainType": { "type": "relation", "relation": "oneToOne", "target": "api::strain-type-copy.strain-type-copy" }, "otherInformation": { "type": "text" }, "genetics": { "type": "string" }, "tags": { "type": "relation", "relation": "oneToMany", "target": "api::tag.tag" }, "weightOverwrite": { "type": "string", "regex": "^\\S*$" }, "activeTerpenes": { "type": "component", "repeatable": true, "component": "product.active-terpene" }, "strapiName": { "type": "string", "private": true } } } This is the hit { "id": "inventory-3492", "productERPId": "6182c40d82284f000134e060", "createdAt": "2022-09-03T15:30:41.651Z", "updatedAt": "2022-11-09T06:35:07.782Z", "publishedAt": "2022-09-03T15:30:41.647Z", "quantity": 10, "priceMed": 50, "priceRec": 50, "specialPriceMed": 20, "specialPriceRec": 20, "ERPSyncedAt": "2022-11-09T06:35:07.774Z", "ordering": 138, "weight": ".5g", "thcMg": 408, "cbdFormatted": "0.25%", "thcFormatted": "81.6%", "inventoryMenuType": "BOTH", "retailer": { "id": 7, "nameCopy": "Liberty Health Sciences - Bradenton", "ERPId": "7c0f4f7e-df0c-4d13-8e52-057828d593e1", "slug": "liberty-health-sciences-bradenton", "street": "1475 Cortez Rd W", "city": "Bradenton", "state": "FL", "zipcode": "34207", "latitude": 27.4620704, "longitude": -82.5767994, "createdAt": "2022-01-07T09:38:54.858Z", "updatedAt": "2022-11-09T06:35:09.845Z", "publishedAt": "2022-08-26T14:59:45.913Z", "ERPSyncedAt": "2022-11-09T06:35:09.841Z", "hours": { "pickup": { "Friday": { "end": "8:30 PM", "start": "9:00 AM", "active": true }, "Monday": { "end": "8:30 PM", "start": "9:00 AM", "active": true }, "Sunday": { "end": "5:00 PM", "start": "10:00 AM", "active": true }, "Tuesday": { "end": "8:30 PM", "start": "9:00 AM", "active": true }, "Saturday": { "end": "8:30 PM", "start": "9:00 AM", "active": true }, "Thursday": { "end": "8:30 PM", "start": "9:00 AM", "active": true }, "Wednesday": { "end": "8:30 PM", "start": "9:00 AM", "active": true } }, "special": null, "delivery": { "Friday": { "end": "00:00 AM", "start": "00:00 AM", "active": false }, "Monday": { "end": "00:00 AM", "start": "00:00 AM", "active": false }, "Sunday": { "end": "00:00 AM", "start": "00:00 AM", "active": false }, "Tuesday": { "end": "00:00 AM", "start": "00:00 AM", "active": false }, "Saturday": { "end": "00:00 AM", "start": "00:00 AM", "active": false }, "Thursday": { "end": "00:00 AM", "start": "00:00 AM", "active": false }, "Wednesday": { "end": "00:00 AM", "start": "00:00 AM", "active": false } } }, "phone": null, "customerType": "Medical", "specialsTitle": null, "specialsUrl": null, "email": null, "details": null, "about": null }, "product": { "id": 135, "nameCopy": "Secret Orchard - Blue Raspberry Vape Cartridge", "ERPId": "174946", "slug": "secret-orchard-blue-raspberry-vape-cartridge", "description": "Cannabis vaporizers are a great way to consume discreetly and consistently. Vape cartridges contain concentrated cannabis oil that is heated by a battery and vaporized for inhalation. These products are very potent and are designed to be consumed in 2-3 second puffs.", "onset": null, "duration": null, "relaxed": 50, "happy": 50, "focused": 50, "hungry": 50, "sleepy": 50, "painRelief": 50, "createdAt": "2022-01-10T20:12:06.933Z", "updatedAt": "2022-08-04T15:25:22.565Z", "ordering": 138, "coaUrl": null, "publishedAt": "2022-01-10T20:12:06.933Z", "otherInformation": null, "genetics": null, "weightOverwrite": null, "strapiName": "Secret Orchard - Blue Raspberry Vape Cartridge" }, "activeTerpenes": [], "seo": null, "specials": [ { "id": 697395, "name": "$20 .5g Cartridges", "ERPId": "63692abfe5d18700ccf7dbe5", "type": "SALE", "menuType": "BOTH", "displayName": null, "description": "Excludes Haze .5g cartridges", "startStamp": "2022-11-08T02:00:00.000Z", "endStamp": "2022-11-10T01:30:00.000Z", "days": null, "createdAt": "2022-11-08T02:02:02.363Z", "updatedAt": "2022-11-08T02:02:02.363Z" }, { "id": 697396, "name": "$35 1g Distillate Cartridges and Syringes", "ERPId": "636987da76cf4400e2b7c973", "type": "SALE", "menuType": "BOTH", "displayName": null, "description": "Excludes Haze Live Resin Cartridges", "startStamp": "2022-11-08T02:00:00.000Z", "endStamp": "2022-11-10T01:30:00.000Z", "days": null, "createdAt": "2022-11-08T02:02:02.397Z", "updatedAt": "2022-11-08T02:02:02.397Z" } ], "_formatted": { "id": "inventory-3492", "productERPId": "6182c40d82284f000134e060", "createdAt": "2022-09-03T15:30:41.651Z", "updatedAt": "2022-11-09T06:35:07.782Z", "publishedAt": "2022-09-03T15:30:41.647Z", "quantity": "10", "priceMed": "50", "priceRec": "50", "specialPriceMed": "20", "specialPriceRec": "20", "ERPSyncedAt": "2022-11-09T06:35:07.774Z", "ordering": "138", "weight": ".5g", "thcMg": "408", "cbdFormatted": "0.25%", "thcFormatted": "81.6%", "inventoryMenuType": "BOTH", "retailer": { "id": "7", "nameCopy": "Liberty <ais-highlight-0000000000>H</ais-highlight-0000000000>ealth Sciences - Bradenton", "ERPId": "7c0f4f7e-df0c-4d13-8e52-057828d593e1", "slug": "liberty-<ais-highlight-0000000000>h</ais-highlight-0000000000>ealth-sciences-bradenton", "street": "1475 Cortez Rd W", "city": "Bradenton", "state": "FL", "zipcode": "34207", "latitude": "27.4620704", "longitude": "-82.5767994", "createdAt": "2022-01-07T09:38:54.858Z", "updatedAt": "2022-11-09T06:35:09.845Z", "publishedAt": "2022-08-26T14:59:45.913Z", "ERPSyncedAt": "2022-11-09T06:35:09.841Z", "hours": { "pickup": { "Friday": { "end": "8:30 PM", "start": "9:00 AM", "active": true }, "Monday": { "end": "8:30 PM", "start": "9:00 AM", "active": true }, "Sunday": { "end": "5:00 PM", "start": "10:00 AM", "active": true }, "Tuesday": { "end": "8:30 PM", "start": "9:00 AM", "active": true }, "Saturday": { "end": "8:30 PM", "start": "9:00 AM", "active": true }, "Thursday": { "end": "8:30 PM", "start": "9:00 AM", "active": true }, "Wednesday": { "end": "8:30 PM", "start": "9:00 AM", "active": true } }, "special": null, "delivery": { "Friday": { "end": "00:00 AM", "start": "00:00 AM", "active": false }, "Monday": { "end": "00:00 AM", "start": "00:00 AM", "active": false }, "Sunday": { "end": "00:00 AM", "start": "00:00 AM", "active": false }, "Tuesday": { "end": "00:00 AM", "start": "00:00 AM", "active": false }, "Saturday": { "end": "00:00 AM", "start": "00:00 AM", "active": false }, "Thursday": { "end": "00:00 AM", "start": "00:00 AM", "active": false }, "Wednesday": { "end": "00:00 AM", "start": "00:00 AM", "active": false } } }, "phone": null, "customerType": "Medical", "specialsTitle": null, "specialsUrl": null, "email": null, "details": null, "about": null }, "product": { "id": "135", "nameCopy": "Secret Orchard - Blue Raspberry Vape Cartridge", "ERPId": "174946", "slug": "secret-orchard-blue-raspberry-vape-cartridge", "description": "Cannabis vaporizers are a great way to consume discreetly and consistently. Vape cartridges contain concentrated cannabis oil that is <ais-highlight-0000000000>h</ais-highlight-0000000000>eated by a battery and vaporized for inhalation. These products are very potent and are designed to be consumed in 2-3 second puffs.", "onset": null, "duration": null, "relaxed": "50", "happy": "50", "focused": "50", "hungry": "50", "sleepy": "50", "painRelief": "50", "createdAt": "2022-01-10T20:12:06.933Z", "updatedAt": "2022-08-04T15:25:22.565Z", "ordering": "138", "coaUrl": null, "publishedAt": "2022-01-10T20:12:06.933Z", "otherInformation": null, "genetics": null, "weightOverwrite": null, "strapiName": "Secret Orchard - Blue Raspberry Vape Cartridge" }, "activeTerpenes": [], "seo": null, "specials": [ { "id": "697395", "name": "$20 .5g Cartridges", "ERPId": "63692abfe5d18700ccf7dbe5", "type": "SALE", "menuType": "BOTH", "displayName": null, "description": "Excludes <ais-highlight-0000000000>H</ais-highlight-0000000000>aze .5g cartridges", "startStamp": "2022-11-08T02:00:00.000Z", "endStamp": "2022-11-10T01:30:00.000Z", "days": null, "createdAt": "2022-11-08T02:02:02.363Z", "updatedAt": "2022-11-08T02:02:02.363Z" }, { "id": "697396", "name": "$35 1g Distillate Cartridges and Syringes", "ERPId": "636987da76cf4400e2b7c973", "type": "SALE", "menuType": "BOTH", "displayName": null, "description": "Excludes <ais-highlight-0000000000>H</ais-highlight-0000000000>aze Live Resin Cartridges", "startStamp": "2022-11-08T02:00:00.000Z", "endStamp": "2022-11-10T01:30:00.000Z", "days": null, "createdAt": "2022-11-08T02:02:02.397Z", "updatedAt": "2022-11-08T02:02:02.397Z" } ] } }

As you can see product does not contain the image object but if I index Product the image object is there

hamzaahmed074 commented 2 years ago

how can I receive information from the nested nested field because only first level information is in the nested fields not second level

bidoubiwa commented 2 years ago

Hey @hamzaahmed074 , what is your populate rule ? Did you try it directly using the strapi API to ensure it works ?

hamzaahmed074 commented 2 years ago

Hey @hamzaahmed074 , what is your populate rule ? Did you try it directly using the strapi API to ensure it works ?

I have not set any populate rule can you tell me how and where to set those ?

bidoubiwa commented 2 years ago

Using this setting: https://github.com/meilisearch/strapi-plugin-meilisearch#-entries-query you can add a populate field which contains the rules to fetch your nested objects.

https://docs.strapi.io/developer-docs/latest/developer-resources/database-apis-reference/entity-service/populate.html#basic-populating

hamzaahmed074 commented 2 years ago

@bidoubiwa It works thanks can you guy update the docs as this was hard to find

bidoubiwa commented 2 years ago

See #580 suggesting more information on populate. Closing as it is resolved :)