nuxt-hub / core

Build full-stack applications with Nuxt on CloudFlare, with zero configuration.
https://hub.nuxt.com
Apache License 2.0
951 stars 54 forks source link

[Question] Strategy for invalidating cache #289

Open remihuigen opened 1 month ago

remihuigen commented 1 month ago

I was wondering about the proper implementation of cache invalidation. Basically, I want to invalidate certain cached functions, when triggering new builds of external apps.

To achieve this I've got this task, which is similar to the Nitro docs: https://nitro.unjs.io/guide/cache#cache-keys-and-invalidation

export default defineTask({
    meta: {
      name: "content:purge",
      description: "Purge all content related cache for a fresh build",
    },
    async run() {
        console.log('purging cache....')

        const cacheKeys = await useStorage('cache').getKeys().then(keys => keys.filter(key => {
          return  key.startsWith('nitro:functions') &&
                  !key.startsWith('nitro:functions:find') &&
                  !key.includes('onderwijsloket') &&
                  !key.includes('Onderwijsloket')
        }))

        cacheKeys.forEach(async key => {
          await useStorage('cache').removeItem(key)
        })

        console.log('✨ cache purge done. ', `Removed ${cacheKeys.length} cached items`)
        return { result: "Success" };
    },
  });

I've also got an api endpoint that programmatically runs this task

export default defineEventHandler({
  onRequest: [
      authTasks
  ],
  handler: async(event) => {
    const { result } = await runTask("content:purge");
    return { result };
  }
})

This works perfectly fine in local dev, but does nothing when running in a CF environment (neither in preview nor production).

Am I missing something here?

atinux commented 3 weeks ago

Hey @remihuigen

Right now the Nitro tasks are not available in production (that's why they are still experimental) as we need to have a way to protect them.

What you can do in the meantime is to use an API route for this that you protect yourself using a Authorization header for instance.

remihuigen commented 2 weeks ago

What you can do in the meantime is to use an API route for this that you protect yourself using a Authorization header for instance.

That's what I'm doing with this endpoint, isn't it?

export default defineEventHandler({
  onRequest: [
      authTasks // Middleware to protect tasks
  ],
  handler: async(event) => {
    const { result } = await runTask("content:purge");
    return { result };
  }
})

I'm using this setup for a bunch of tasks, with a external cronjobs for fetching these endpoints, and this works perfectly (in production).

My issue is with

       const cacheKeys = await useStorage('cache').getKeys().then(keys => keys.filter(key => {
          return  key.startsWith('nitro:functions') &&
                  !key.startsWith('nitro:functions:find') &&
                  !key.includes('onderwijsloket') &&
                  !key.includes('Onderwijsloket')
        }))

        cacheKeys.forEach(async key => {
          await useStorage('cache').removeItem(key)
        })

where const cacheKeys = await useStorage('cache').getKeys() seems to be returning an empty array in production.. And i have no clue why.

When i visit admin.hub.nuxt.com/{team}/{project}/production/server/cache there's a bunch a cache keys.

I'm probably missing something obvious here...