unjs / unstorage

💾 Unstorage provides an async Key-Value storage API with conventional features like multi driver mounting, watching and working with metadata, dozens of built-in drivers and a tiny core.
https://unstorage.unjs.io
MIT License
1.86k stars 138 forks source link

Cached Storage #15

Open pi0 opened 3 years ago

pi0 commented 3 years ago

(some good ideas in https://github.com/unjs/unstorage/issues/88)

chris-visser commented 2 years ago

Not sure wjat you mean with this, but I had this idea to allow setting the cache via s3 endpoints and then expose it via a cloudfront distribution.

I'm currently attempting to create the s3 driver. Wdyt of adding a cdn option to drivers like s3 and maube other bucket providers?

It would be something like this:

driver({ 
  bucket: 'my-site',
  region: 'eu-west-1',
  basePath: '/cache',
  acl: 'public-read',
  ...iamCredentials,
  cloudfrontDistribution: { ... } 
}
tanishqmanuja commented 1 week ago

For now I am using something like this (for similar problem to #379 ), are there any downsides to this approach ?

import { setTimeout } from "node:timers/promises";

import { createStorage } from "unstorage";
import { fsCacheDriver } from "./driver";

const storage = createStorage({
  driver: fsCacheDriver({ base: "./db", freshness: "5s" }),
});

await storage.setItem("foo", "bar");

let foo = await storage.getItem("foo");
console.log(foo); // output - bar

await setTimeout(5000);

foo = await storage.getItem("foo");
console.log(foo); // output - undefined
driver.ts ```ts import fsLiteDriver, { type FSStorageOptions } from "unstorage/drivers/fs-lite"; import { normalize, type Duration } from "./duration"; export interface FSCacheStorageOptions extends FSStorageOptions { freshness?: Duration; } const DRIVER_NAME = "fs-cache"; const DEFAULT_EXPIRATION = "30m"; export const fsCacheDriver = (options: FSCacheStorageOptions = {}) => { const driver = fsLiteDriver(options); const freshness = normalize(options.freshness ?? DEFAULT_EXPIRATION); return { ...driver, name: DRIVER_NAME, options, async getItem(key: string, opts: { freshness?: Duration } = {}) { const item = await driver.getItem(key); if (item === null || item === undefined) { return item; } const meta = await driver.getMeta?.(key, {}); if (meta === null || meta === undefined || meta.mtime === undefined) { return item; } const etime = opts.freshness ? normalize(opts.freshness) : freshness; if (Date.now() - new Date(meta.mtime).getTime() < etime) { return item; } if (!options.noClear) { await driver.removeItem?.(key, {}); } }, }; }; ```
duration.ts ```ts export const UNITS = { s: 1000, m: 60000, h: 3600000, d: 86400000, w: 604800000, } as const; export type Unit = keyof typeof UNITS; export type DurationWithUnit = `${number}${Unit}`; export type Duration = DurationWithUnit | number; export function normalize(duration: Duration): number { if (typeof duration === "number") { return duration; } const value = duration.slice(0, -1); const unit = duration.slice(-1); const multiplier = UNITS[unit as Unit]; if (!multiplier) { throw new Error(`Invalid unit ${unit}`); } return Number(value) * multiplier; } ```