Producer list very slow in loading

[mariocarabotta]

sometimes it never loads the list sometimes it loads at about 1-2minutes (Mario)

[mkllnk]

We moved the API workflows to a new n8n server which is not crashing any more. And we moved the proxy cache to the OFN AU production server which is a lot faster. I also added some caching headers for the browser.

A page load now takes between 7 and 13 seconds on my machine. It's not ideal but the best we can do for those 800 requests. Once we limit do the verified data, this will be a lot faster.

Is this good enough?

Otherwise we have to meet with SiB to see if there is any way to reduce the number of requests.

[mariocarabotta]

same here with the SiB test environment. WP website still at 1 minute loading time, but maybe it's an old version of the component?

I think 7 second it's ok for launch. it's not super high performance, but maybe by preloading the component in the background we might save some time?

@kirstenalarsen looping you in this one

[mkllnk]

I just tried again on our Wordpress page. The first load took 2 minutes. A second load without browser cache took 17 seconds. So it looks like the proxy server is not serving old cached responses after a while. I need to look into that again at some point but once the page is busier, the cache should work better and not become stale that often.

[mkllnk]

I asked JB about optimising the loading and he's suggesting to consolidate more data in one document:

A huge improvement would be to serialize everything on the first request (/webhook/regen/producers), the more you serialize there the less we have to request. But you have to ensure that every nodes are included.

So let's see where we can consolidate data. Data from one page load:

• 288 XHR GET requests total, of that:
• 110 assurance
• 53 products
• 44 producer
• 42 address
• 29 context.json (cached)

The biggest impact would be to include assurance objects within the enterprise endpoint.

The other approach would be to have another script to crawl all endpoints, resolve all objects and merge them in one object. Then we can provide that one document. :thinking:

[mkllnk]

So I got a couple of endpoints which aggregate all the data but the component doesn't seem to handle it well and raises an error. I'll see what JB says to that.

[kirstenalarsen]

@JbPasquier ping re. above . .

[mkllnk]

Progress!

I created two new endpoints to pre-load all the data and it seems to work very nicely. In a private window, if I go to the homepage first and then click on the directory, it takes 9.2 seconds to render completely. But some data is shown much earlier.

[mkllnk]

I added a new Wordpress plugin to add code snippets to the header. They now load the SiB components on any page and pre-load the directory data.

[kirstenalarsen]

Ace! The products page is not loading for me this morning - might this be something to do with this or the product mapping?

On Thu, 18 Apr 2024 at 13:32, Maikel @.***> wrote:

I added a new Wordpress plugin to add code snippets to the header. They now load the SiB components on any page and pre-load the directory data.

— Reply to this email directly, view it on GitHub https://github.com/openfoodfoundation/sib-discovery-components/issues/39#issuecomment-2062927981, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAWGXSHXKEBTYUFPMFWUIOLY545F5AVCNFSM6AAAAABFUKPHSSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRSHEZDOOJYGE . You are receiving this because you were mentioned.Message ID: @.*** com>

[mkllnk]

The products page is not loading for me this morning

Your wifi dropped out this morning. Did you have any more problems later on?

[kirstenalarsen]

I think that as long as I'm not the person who hits the cache reload it's ace :) Great work on this @mkllnk and @rioug - performance is now excellent 🙏