Feature/afosto-plugin

[gijsbotje]

Afosto plugin to retrieve data from a webshop.

Enter the shop url in the configuration and use the fieldtypes to get products and categories

[gijsbotje]

@samijaber Thanks for checking it. What is it you expect the categories to return? I haven't created a component for a category yet, so I haven't thoroughly tested it. If the issue is that it doesn't show options when the field is opened, that's just how our suggest endpoint works. It needs a q value to find pages/collections. Right now it just fetches data from the actual shop, in the future it should use an API, but we aren't there yet. It also returns blank when you do the same request within, I think, 300ms.

When I test the field Afosto category handle I can search for pages and select and save it.

I did add a header to the getRequestObject functions to fetch the JSON instead of HTML. Maybe that was what you were missing :)

[gijsbotje]

@samijaber do you have some documentation of what is possible with the getRequestObject function? could I also transform the response to something that would be a stripped version of what is returned now?

[gijsbotje]

@samijaber Any estimate when this will be reviewed again?

[samijaber]

Hey @gijsbotje, I was away for the past 2 weeks, hence the slowness in responses. Thanks for your patience!

Here's a screencast outlining what I'm seeing: https://www.loom.com/share/4b23a91b53e4413fb33299ceb6dbfb4b

I think one confusing thing for us is that the same API endpoint is used to search for both products & collections. It looks like, if q is empty, then it return all products, but if it has a q= value, then it returns a filtered set of collections. We would expect 2 separate endpoints, each allowing to search (or get all) values for both products & collections.

do you have some documentation of what is possible with the getRequestObject function? could I also transform the response to something that would be a stripped version of what is returned now?

We don't currently have that documented. I will work on getting some documentation up and running for you soon.

[gijsbotje]

@samijaber I understand the confussion. As i mentioned before, we currently don't have api's to search for them separately. That is why I am using our search suggest endpoint from the site directly. This endpoint returns pages and products, but has some bugs that you encountered. For now it is our only way of getting to the data we need.

Of course we want a better solution, but we aren't able to do that right now. We would like to move forward with this solution so we can use it for our customers.

[gijsbotje]

@samijaber I've created tasks for my colleague to take a look at what we can change to better suite your expectations. Will get back to you when/if we change it.

[gijsbotje]

@samijaber We tweaked the endpoint a little, could you guys check it out?

[samijaber]

@gijsbotje Thanks! I just tested it on my end, and the search seems to work properly for both products & categories! 👍🏽

[samijaber]

@gijsbotje I just took another look and tried to use your plugin in a demo Builder space. Searching for the data works perfectly!

But when it comes to using the data in the editor, there is one big issue: the requests stored by the plugin return very large product & category files. I tried a few as examples and they were 2.5-3mb in size. Unfortunately, this makes the dynamic data feature really impractical to use with Afosto data.

The reason for that is that our data bindings code will go through all of the data, and make every single value in you data a potential item in the list. There are about 50,000-60,000 values in each one of your products/categories.

I have recorded a loom to show you: I start by using the contentstack plugin to show how other plugins work, and then tried the Afosto plugin. https://www.loom.com/share/f02f1cc6ef684a88a76f7ca4ffcc0ab5

I'm not sure how you want to approach resolving this on your end. Ultimately, the URLs stored by your plugin need to return much smaller pieces of content, otherwise, the plugin won't be practical to use. 🤔

For reference, here are examples of the URLs that I am talking about (in curl form). They're the values generated by getRequestObject:

• product: curl --request GET --url 'https://www.azalp.nl/keter-17192190-manor-43-2e-kans-128-103-1-2.htm' --header 'Content-Type: application/json'
• category: curl --request GET --url 'https://www.azalp.nl/filter-flosser/' --header 'Content-Type: application/json'

[mhevery]

Hey, there is no activity on this PR. Can we get this PR into mergeable state (or should we abandon it and close it)