vlio20 / utils-decorators

Decorators for web and node applications
https://vlio20.github.io/utils-decorators/
MIT License
216 stars 13 forks source link

Add support to gettings the method name or expirationTimeMs in cach #139

Closed AlenToma closed 2 years ago

AlenToma commented 2 years ago

Hi Again.

I was thinking while implementing my own cache, that it would be great to be able to get expirationTimeMs in the cache class as a parameters.

As of now I can only specify how much the cache would be contained globally for the whole cache class.

See here example of my own cache


export class AsyncFileCustomCache implements AsyncCache<any>
{
    private reader: Reader;
    private path: string;
    constructor() {
        this.reader = new Reader();
        this.path = ((FileSystem.cacheDirectory || FileSystem.documentDirectory || "") + "JSONTemp").replace("file:///", "")
    }

    async validate() {
        if (!(await this.reader.exists(this.path)))
            await this.reader.mkdir(this.path);
    }

    getKey = (key: any) => {
        let k = "";
        if (Array.isArray(key))
            k = key.map(x => x.toString()).join("");
        else k = key.toString();

        return k.replace(/(\/|-|\.|:|"|'|\{|\}|\[|\]|\,| |\’)/gmi, "").toLowerCase() + ".jpg";
    }

    getPath = (key: string) => {
        var k = this.getKey(key);
        if (k.isEmptyOrSpaces()) {
            console.error("Obs Key is empty", key)
            return undefined;
        }
        return this.path + "/" + k
    }

    set = async (key: string, parameter: any) => {
        try {
            let k = this.getPath(key);
            if (k === undefined || !parameter)
                return;
            await this.validate();
            await this.reader.writeFile(k, JSON.stringify({ date: new Date(), data: parameter } as DataCache));
        } catch (e) {
            console.error(e);
            await this.delete(key);
        }
    }
    get = async (key: string) => {
        try {
            let k = this.getPath(key);

            if (k === undefined)
                return null;
            await this.validate();
            const jsonData = JSON.parse(await this.reader.readFile(k)) as DataCache;
            var date = new Date(jsonData.date)
            console.log("Keys", Object.keys(jsonData.data))
            if (!jsonData.data || date.days_between() >= 5)
                await this.delete(key);
            return jsonData.data;
        } catch (e) {
            console.error(e);
            await this.delete(key);
            return null;
        }
    }
    delete = async (key: string) => {
        await this.validate();
        let k = this.getPath(key);
        if (k === undefined)
            return;
        if (await this.has(key))
            await this.reader.unlink(k)
    }
    has = async (key: string) => {
        let k = this.getPath(key);
        if (k === undefined)
            return false;
        var hasKey = await this.reader.exists(k);
        return hasKey;
    }
}

See in get I have date.days_between() it would have been great if I could have getting access to expirationTimeMs or the methods name at least in it.

What do you think?

vlio20 commented 2 years ago

@AlenToma why not provide the ms to your cache when creating it, something like this:

export class AsyncFileCustomCache implements AsyncCache<any> {
    private reader: Reader;
    private path: string;
    constructor(private readonly ms: number) {
      .....
    }
}

@asyncMemoize({
   cache: new AsyncFileCustomCache(5000)
})
someAsyncMethod(): Promise<any> {
   ....
}

You don't have to provide the decorator expirationTimeMs in that case. Will that work for you?

AlenToma commented 2 years ago

Yes that it is exactly what I did, But there is more to this problem.

Able to pass parameters or the target to the cache, is a really smart useful thing.

Will give you an example here below and let me know what you think.

type Parser ={
  name: string;
  getData: (dataname: string)=> string;
}

class TestParser implements Parser {
  name: string;
  constructor(name: string){
    this.name = name;
  }

  @asyncMemoize({
   cache: new AsyncFileCustomCache(5000)
  })
  getData(dataname: string){
    // return some data
  }
}

var parsers = [new TestParser("parser1"), new TestParser("parser2")]

const data1= parsers[0].getData("testo");
const data2= parsers[1].getData("testo");

See above there is two different object with two different data that could contain the same dataname. The Ideal scenario will be that the key should contain the name and dataname. But I cant do that unless I pass name to getData for which I do not like to pass something I do not use in the method it self.

So passing the target, eg the current TestParser to the cache is really helpfull.

Now This is not the original issue I created this issue for, But I came after this issue a little late.

I could open another issue with this if you so think this is a useful thing to implement.

vlio20 commented 2 years ago

I think that this is a different issue. But if you want to have control over the cache key you could also provide a keyResolver to the decorator:

  @asyncMemoize({
   cache: new AsyncFileCustomCache(5000),
   keyResolver:(dataname: string) => {
      return 'YOUR_PREFIX' + dataname;
   }
  })
  getData(dataname: string){
    // return some data
  }

Will that work?

AlenToma commented 2 years ago

That wont work becouse i wont have access to testparser, aka this in key resolver.

The key should in this case be name + dataname

vlio20 commented 2 years ago

@AlenToma, not sure what you are trying to achieve. Do you want to have the same cache be shared between the 2 instances of the parser?

AlenToma commented 2 years ago

Yes. The cache should include the current testparser.

vlio20 commented 2 years ago

Why not create it on the app level and then use it on every instance?

const myCache = new AsyncFileCustomCache(5000);

class SomeClass {
  @asyncMemoize({
   cache: myCache
  })
  getData(dataname: string){
    // return some data
  }
}
AlenToma commented 2 years ago

That is not the issue. Nevermind. Close this if youvsee this as not impotend

AlenToma commented 2 years ago

@vlio20 I created my own instead to use it as file caching

See Below, you may get some ideas on how to improve the library and also understand what I mean.

const getKey = (option: {
    cache: FileMemoCache,
    daysToSave: number,
    identifierPropertyName?: string
}, propertyName: string, target: any, ...args: any[]) => {
    let key = JSON.stringify(args) + propertyName;
    if (option.identifierPropertyName !== undefined)
        key += target[option.identifierPropertyName];
    return key.replace(/(\/|-|\.|:|"|'|\{|\}|\[|\]|\,| |\’)/gmi, "").toLowerCase() + ".jpg";
}

const callingFun = new Map<string, boolean>();

export default function FileMemo(option: {
    cache: FileMemoCache,
    daysToSave: number,
    identifierPropertyName?: string,
    validator?: (params: any) => boolean;
}) {
    return function (target: any, propertyKey: string, descriptor: PropertyDescriptor) {
        const currentFn = descriptor.value as (...args: any[]) => Promise<any>;
        descriptor.value = async function (...args: any[]) {
            const key = getKey(option, propertyKey, this, args);
            while (callingFun.has(key))
                await httpClient.wait(10);
            let data = null as DataCache | null;
            callingFun.set(key, true)
            try {
                if (await option.cache.has(key)) {
                    data = await option.cache.get(key);
                }

                if (data && typeof data.date === "string")
                    data.date = new Date(data.date);

                if (!data || data.date.days_between() >= option.daysToSave) {
                    try {
                        let data2 = await currentFn.bind(this)(...args);
                        if (data2) {
                            if (!option.validator || option.validator(data2)) {
                                if (data)
                                    await option.cache.delete(key);
                                await option.cache.set(key, { date: new Date(), data: data2 });
                                return data2;
                            } else {
                                if (data) {
                                    await option.cache.delete(key);
                                    data.date = new Date();
                                    await option.cache.set(key, data); // extend the date
                                }
                                return data?.data ?? data2;
                            }
                        }
                    } catch (e) {
                        console.error("FileMemo", e);
                    }
                }
                return data?.data;
            } finally {
                callingFun.delete(key)
            }
        }

    }
}