Instantiating MemoryCache takes ~ 40 seconds locally and leads to wrangler script timeouts

[styxlab]

Describe the bug

Instantiating the MemoryCache (lru-cache) in the loader config takes more than 40 seconds on my fast desktop. I also can't publish with wrangler publish as I am getting a script timeout due to the same reason. When I remove the cache property in the config object, everything works just fine. What could be the reason and how do I take advantage of the MemoryCache without timing out?

Thanks for this awesome repo and documentation, this is an outstanding project!

Steps to Reproduce the Bug or Issue

1. npx create-remix remix-image --template cloudflare-workers
2. cd remix-image; yarn add remix-image @next-boost/hybrid-disk-cache
3. add routes/api/image.ts according to https://remix-image.mcfarl.in/docs/tutorial-extras/cloudflare#transformer with maxSize: 5e7
4. Add image component

Expected behavior

No timeouts, fast startups, working wrangler publish.

Platform

• OS: Linux localhost.localdomain 5.16.18-200.fc35.x86_64 SMP PREEMPT Mon Mar 28 14:10:07 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
• Browser: Chrome, Firefox

[elliotlaws]

I'm experiencing the same using Cloudflare Pages.

[styxlab]

I abandoned the LRU cache and changed it to KV. It's a good solution if you are on Cloudflare.

[elliotlaws]

I abandoned the LRU cache and changed it to KV. It's a good solution if you are on Cloudflare.

Nice, do you have an example of the loader you're using with KV?

[styxlab]

Nice, do you have an example of the loader you're using with KV?

Still work in progress, but you can find it here: https://github.com/styxlab/remix-image-cloudflare.

[Josh-McFarlin]

Looks like a great package @styxlab! Apologies for the delay in my response, I've been incredibly busy. Unfortunately, I cannot reproduce this on my machine, but I am still looking into this.

MemoryCache uses lru-cache for its implementation. Its a great library and has a lot of test cases written, so I believe the problem is likely in my implementation or something with Remix's server.

One possible cause I see is that I forgot a field in the options:

// function to calculate size of items.  useful if storing strings or
// buffers or other items where memory size depends on the object itself.
// also note that oversized items do NOT immediately get dropped from
// the cache, though they will cause faster turnover in the storage.
sizeCalculation: (value, key) => {
  // return an positive integer which is the size of the item,
  // if a positive integer is not returned, will use 0 as the size.
  return 1
},

If you would like to try implementing this yourself, you should copy the MemoryCache code and change the constructor to include something along the lines of:

this.cache = new LRU<string, Uint8Array>({
  max: this.config.maxSize,
  ttl: this.config.ttl,
  allowStale: true,
  updateAgeOnGet: true,
  sizeCalculation: (value, key) => value.byteLength,
});

However, I'm not sure this is the cause of the issue. One problem I ran across while developing this package was Remix's request purging in development. Since it clears itself on each request, MemoryCache gets emptied on every request in development. Since instantiating MemoryCache is taking so long, I believe this is most likely the error as it's happening at startup instead of a filled cache mentioned in the fix above. Its possible the request purging is causing the cache to reset (and maybe loop) during instantiation, causing some issues. This would likely prevent building.

I will look into this over the next week and hopefully find a fix.

[styxlab]

Thanks for your suggestions, will revisit the issue next week.

[Josh-McFarlin]

@styxlab @elliotlaws Today I released v1.0.0 of Remix Image and worked on MemoryCache. It appears to work correctly on Cloudflare environments in dev, but it would be great if you could try the updates and let me know if the issues persist. Thanks!