Improve load time for listings on the Canadian directory

[David-OFN-CA]

@Louiscsn can SiB help us fix the load time issues that we are encountering with the Waterloo Directory at present? Here's a link the site, for your reference:

https://waterlooregionfood.ca/staging/5285/

It's taking about 5 seconds to load 5 entries

Can you do a little troubleshooting on your end, and see if the problem is being caused on your side, or whether we need to engage OFN devs to assist with as well?

[JbPasquier]

Related to the server side bundling, see note there https://github.com/openfoodfoundation/sib-discovery-components/issues/66#issuecomment-2271188760

You have to enable those routes and configure them on your wordpress.

[David-OFN-CA]

@mkllnk @rioug could either of you assist us with a load time issue that JB thinks is related to server side bundling?

To address our load time issues JB is suggesting that the Aus team share with @RebeccaOFN the the htaccess file that you are using in the deployment of the Disco Regen wordpress site

Rebecca requested access to the file a few weeks back, in issue 66, under the dicussion that starts here:

https://github.com/openfoodfoundation/sib-discovery-components/issues/66#issuecomment-2271188760

[mkllnk]

@RebeccaOFN, the site speed has nothing to do with the .htaccess file. It's about bundling all data in one file. We have a workflow for that and I just updated it to include the Canadian data. Replace these URLs:

• https://api.regenerative.org.au/webhook/regen_CAN/producers with https://api.regenerative.org.au/webhook/regen_CAN/producer-bundle
• https://api.regenerative.org.au/webhook/regen_CAN/products with https://api.regenerative.org.au/webhook/regen_CAN/product-bundle

From now on, with this bundle, it will take between 30 and 60 minutes for any updates to come through because the bundle is cached.

[mkllnk]

I now followed the other discussion around the htaccess file. That is to view a producer profile by accessing the URL directly. I put the following part outside of the Wordpress block so that it doesn't get overridden by Wordpress:

# Startin'Blox component routing:
RewriteEngine On
RewriteBase /
RewriteRule ^directory.*$ index.php?pagename=directory [L]

This assumes that the Wordpress page with the component is called directory. In your case it seems to be staging at the moment. But you can also try to add that:

RewriteRule ^directory.*$ index.php?pagename=directory [L]

Just add RewriteRule lines for every page name you want to work for this.

[David-OFN-CA]

Thanks a lot @mkllnk - much appreciated!

@RebeccaOFN are you able to move forward with this, or do we need to touch base?

[RebeccaOFN]

@RebeccaOFN, the site speed has nothing to do with the .htaccess file. It's about bundling all data in one file. We have a workflow for that and I just updated it to include the Canadian data. Replace these URLs:

• https://api.regenerative.org.au/webhook/regen_CAN/producers with https://api.regenerative.org.au/webhook/regen_CAN/producer-bundle
• https://api.regenerative.org.au/webhook/regen_CAN/products with https://api.regenerative.org.au/webhook/regen_CAN/product-bundle

From now on, with this bundle, it will take between 30 and 60 minutes for any updates to come through because the bundle is cached.

@mkllnk , does the replacement need to be made in the Wordpress code or in the n8n workflow? I can see where these urls appear in the code, if that's all that needs to be changed. If it needs to be changed in n8n instead/as well, could you point us to the right workflow?

[RebeccaOFN]

@mkllnk , just following up on the question above. Please let us know when you have a moment. Hopefully this is an easy fix :)

[mkllnk]

You just need to do that in the Wordpress code. Replace producers with producer-bundle in the URL and it will load all data to start with.

You can also add this HTML to your homepage:

<link rel="preload" href="https://api.regenerative.org.au/webhook/regen_CAN/producer-bundle " as="fetch" type="application/json" crossorigin/>

It will trigger the download of the bundle in the background when people first visit the website. Then when they want to look at the data it will load much faster because it's already in the browser cache.

[RebeccaOFN]

Awesome, thank you! I will try making those changes today.

[RebeccaOFN]

@mkllnk , I updated the URL and added the HTML but haven't seen any difference in performance. Just checking that I have done both of these correctly. Here is the current code:

<script type="module" src="https://cdn.skypack.dev/@startinblox/core@0.19"></script>
<script type="module" src="https://cdn.skypack.dev/@startinblox/router@0.12"></script>
<script type="module" src="https://cdn.jsdelivr.net/npm/@startinblox/component-ofn-rpp@waterloo"></script>

<solid-ofn-rpp
  producers-src="https://api.regenerative.org.au/webhook/regen_CAN/producer-bundle"
  range-product-categories="https://api.regenerative.org.au/webhook/regen_CAN/produce_categories"
  range-producer-assurances="https://api.regenerative.org.au/webhook/regen_CAN/assurance_partners"
  uniq="regenerative-produce-portal"
  route-prefix="staging/5285"></solid-ofn-rpp>

I added the HTML you posted as an HTML block in Wordpress. Does this need to be done another way?

[David-OFN-CA]

thanks @RebeccaOFN

@mkllnk @kirstenalarsen - jusrt flagging that this performance issue is getting increasingly urgent as we need to go live with the site in under 4 weeks

[mkllnk]

I looked at the page and the pre-loading of the bundle doesn't work because there's an additional space at the end of the URL.

In the HTML it now says:

<link rel="preload" href="https://api.regenerative.org.au/webhook/regen_CAN/producer-bundle%20" as="fetch" type="application/json" crossorigin>

But it should be:

<link rel="preload" href="https://api.regenerative.org.au/webhook/regen_CAN/producer-bundle" as="fetch" type="application/json" crossorigin>

[mkllnk]

Otherwise it seems to load quite quickly for me. One factor may be that the cache server is in Sydney. Maybe we should set up one in Canada so that it's quicker for you there.

[mkllnk]

To know what we are talking about, can you quantify slow? How long does it take to load for you?

Once the files have been loaded into my cache, it takes max 1.5 seconds to see the directory.

[RebeccaOFN]

I looked at the page and the pre-loading of the bundle doesn't work because there's an additional space at the end of the URL.

In the HTML it now says:

<link rel="preload" href="https://api.regenerative.org.au/webhook/regen_CAN/producer-bundle%20" as="fetch" type="application/json" crossorigin>

But it should be:

<link rel="preload" href="https://api.regenerative.org.au/webhook/regen_CAN/producer-bundle" as="fetch" type="application/json" crossorigin>

Okay, I deleted the extra space.

[RebeccaOFN]

To know what we are talking about, can you quantify slow? How long does it take to load for you?

Once the files have been loaded into my cache, it takes max 1.5 seconds to see the directory.

I had been counting it as at least 4-5 seconds, and sometimes longer. It does seem a bit faster than that now.

[mkllnk]

So, shall I invest another hour trying to make it faster with a local cache or are you happy with it for now?

[David-OFN-CA]

Thanks @mkllnk! On my end it's still taking about 4 to 5 seconds to load, I think the local cache soloution you suggest sounds like the way to go - let us know how we can support the process (ping @RebeccaOFN and @Louiscsn for your context)

[JbPasquier]

@David-OFN-CA I tried from BrowserStack, with a fresh environment, it looks like this from there:

https://github.com/user-attachments/assets/3ce9366b-e572-41a8-9ac4-663090cbec82

So, Maikel's track about the fact that the cache server is in Sydney may be a good one to follow.

[David-OFN-CA]

Cheers JB! That's looking nice and quick on your end. Thanks for the quick follow up on this issue - I agree: Maikel's suggestion re setting up a Canadian cahche server looks like our best next step

[mkllnk]

Okay, great. @RebeccaOFN, can you add a subdomain?

Point api.waterlooregionfood.ca to 68.183.204.127 which is your OFN prod server. I can then set the API up there.

[David-OFN-CA]

Thanks @mkllnk! we are intending to get to this today - @RebeccaOFN just checking this is on your radar?

[RebeccaOFN]

@mkllnk I have set up the subdomain (hopefully correctly). Let me know if I need to do anything else/differently.

[mkllnk]

I've set up the caching server. I tried to update the Wordpress page as well but it's not displaying correctly. It's showing the HTML instead of rendering it. I'm not sure what your intention was there and don't want to mess anything up. So I left it as is, just with the URLs replaced.

[David-OFN-CA]

Hi @mkllnk thanks for the update. Since this change, the directory component at https://waterlooregionfood.ca/staging/5285/ no longer works at all.

The directory compoenent at https://waterlooregionfood.ca/staging/5285/directory/, but it still take about 5 seconds to load on my end

Could you advise on next steps?

@RebeccaOFN could you address Maikel's question re the HTML that is showing rather than rendering - or touch base with me if you want to troubleshoot it together, ahead of getting back to Maikel.

cheers all

[RebeccaOFN]

@mkllnk , it looks like you replaced the code block with an html block. I assume that's why it's not working now. Was that on purpose?

@David-OFN-CA , the code on the Directory page hasn't been changed yet. Originally these pages were set up separately so that we could share a version of the directory with the Waterloo Roundtable while troubleshooting the components on the home page. At this point it may make sense to change the structure and eliminate the component test page. I'm not sure.

[RebeccaOFN]

@mkllnk I also haven't been able to get the Rewrite rules working. I tried it as shown and with substituting in 'staging/5285'. Any ideas?

[mkllnk]

it looks like you replaced the code block with an html block. I assume that's why it's not working now. Was that on purpose?

That was not on purpose. I guess that it happened when I opened it in the WP editor because Divi didn't seem to work for me. :shrug:

But let's focus on the current directory page. I fixed the preload of the bundle so that it gets loaded when you reach the homepage. After clicking on the directory, the data is already cached and the page loads quickly for me.

I also added a preload for the new URL for the server closer to you. But I can't edit the directory page with Divi. It just displays the page to me without any editor functionality. Can you change all URLs from api.regenerative.org.au to api.waterlooregionfood.ca?

[mkllnk]

I also haven't been able to get the Rewrite rules working. I tried it as shown and with substituting in 'staging/5285'. Any ideas?

This is a bit tricky to get right. In your example you need these values when calling the component on the directory page:

  uniq="directory"
  route-prefix="staging/5285/directory"

Then the rewrite rule was assuming that you don't have a staging site and that you access the page under . But for your staging site, you need to adjust the rule:

RewriteRule ^staging/5285/directory.*$ index.php?pagename=directory [L]

Or maybe:

RewriteRule ^staging/5285/directory.*$ staging/5285/index.php?pagename=directory [L]

I don't know the Bluehost setup well enough to predict how it's working internally. Any luck with these?

[RebeccaOFN]

it looks like you replaced the code block with an html block. I assume that's why it's not working now. Was that on purpose?

That was not on purpose. I guess that it happened when I opened it in the WP editor because Divi didn't seem to work for me. 🤷

But let's focus on the current directory page. I fixed the preload of the bundle so that it gets loaded when you reach the homepage. After clicking on the directory, the data is already cached and the page loads quickly for me.

I also added a preload for the new URL for the server closer to you. But I can't edit the directory page with Divi. It just displays the page to me without any editor functionality. Can you change all URLs from api.regenerative.org.au to api.waterlooregionfood.ca?

@mkllnk , I changed the URLs but it made the directory stop working altogether (the page loads without the directory). I changed them back to api.regenerative.org.au for the moment.

[mkllnk]

Okay, I found one typo in the cache server configuration. I put the component with the new URLs on the test page:

https://waterlooregionfood.ca/staging/5285/

From Australia, as new visitor, without browser cache, the page needs 3 seconds to just load the HTML, by 5 seconds the component is loading and at 10 seconds you see the directory while some more images are loaded in the background.

A repeat request displays component after 2 seconds which, I think, is good enough for a page with so much data.

I can't tell how quick it will be from Canada but it should be faster when first loading the page. Also, if you first visit the homepage, the component will be pre-loaded. So visiting the component after the homepage should load it within 2 seconds.

Does this work for you?

[mkllnk]

As comparison, this is faster than the Australian directory at the moment because we have a lot more data. On first direct visit, worst case, the directory displays after 12 seconds. A repeat visit displays in 3 seconds.

[David-OFN-CA]

Just updating here that Maikel pushed through a bundling change yesterday that has the directory component running nice and quickly, so I am closing this issue - thanks all for your help on resolving this - we are very happy with where we have landed. For ref https://waterlooregionfood.ca