Storyblok client throttling crashing Astro adapter in CloudFlare Worker

[Soviut]

I have a project using Astro. Astro has a CloudFlare Worker "adapter" for doing server side rendering (SSR). This will re-render the preview page every time someone hits "save" in the Storyblok editor.

However, after several consecutive saves, the worker eventually crashes, giving a "The script will never generate a response" error.

Based on the comment in this thread, it seems to be caused by throttling in the Storyblok client not resolving all its promises and therefore never returning a result. https://github.com/withastro/adapters/issues/43#issuecomment-1782206087

They include links to the code in question as well as some workaround code to monkey patch the Storyblok client.

---

Expected Behavior

I should be able to save frequently without the Storyblok client failing.

Current Behavior

Saving frequently causes the client to eventually fail, crashing the worker.

Steps to Reproduce

The author of the original thread provided a Stackblitz to reproduce https://stackblitz.com/edit/github-tkyzda-4d7fjz?file=astro.config.mjs

1. Create an Astro project
2. Install the CloudFlare adapter for Astro
3. Follow this tutorial to create a generateStaticPaths function for Astro that calls Storyblok https://www.storyblok.com/tp/create-a-preview-environment-for-your-astro-website
4. Deploy to CloudFlare
5. Point Storyblok editor at cloudflare URL
6. Make content changes and save them quickly
7. Observe the cloudflare worker eventually return an error

[thiagosaife]

We'll address this one.

[grantlucas]

I was happy to find this reported issue after going down the exact same path as the original reporter with the exact same set of tools. Is there any estimation on a potential fix for this? Thank you

[patrick-schneider-latori]

@thiagosaife Any updates on this one? We currently omit the client (and created our own fetcher), since it makes all our production sites crash...this should not be the case!

[Soviut]

@patrick-schneider-latori it would be good if you could post your solution or even make a PR with your changes.

[F0rce]

Is there any way to prioritise this? Like some other commenters, we have fallen down this rabbit hole and have not been able to sort it out, before finding this issue.

[negprof]

Also the same problem with this using hydrogen shopify.

[jansvabik]

I have to admit that I've been eagerly waiting for a solution to this problem for months, like many of you. Today, I decided to dive deeper into the code of both this repository and the @storyblok/astro repository to see if I could contribute a solution myself. To my surprise, I discovered that we can adjust the rateLimit parameter in the apiOptions within our astro.config.mjs config. By default, it's set to 5 requests, but adjusting it can significantly impact how the client behaves under heavy load. Here's a quick example of how you can adjust this in your config:

// in your astro.config.mjs
// ...
storyblok({
    accessToken: env.STORYBLOK_TOKEN,
    apiOptions: {
        // ...
        rateLimit: 250,
        // ... 
    },
    // ...
})
// ...

I hope this solution helps others who are encountering the same issue. If anyone has further insights or alternative solutions, I encourage you to share them with the community. 🙂

[remi-freriks]

We have the same problem with Cloudflare Workers. Even with rateLimit at 250. Is Storyblok still working on this problem?