botisan-ai / gpt3-tokenizer

Isomorphic JavaScript/TypeScript Tokenizer for GPT-3 and Codex Models by OpenAI.
MIT License
171 stars 19 forks source link

Unsafe use of `this.cache.hasOwnProperty` #13

Closed kopertop closed 1 year ago

kopertop commented 1 year ago

I started receiving errors about this.cache.hasOwnProperty is not a function. Digging into the code it looks like tokenizer.ts uses a bit of unsafe code considering this.cache is a map that allows any passed in value to be used as a token:

    if (this.cache.hasOwnProperty(token)) {
      return this.cache[token];
    }

Instead, this should be:

    if (Object.prototype.hasOwnProperty.call(this.cache, token)) {
      return this.cache[token];
    }
Pullerz commented 1 year ago

I'm also having this issue, would appreciate it if the PR was merged soon 🙏

lhr0909 commented 1 year ago

@kopertop thank you so much for setting up the PR! Merged

@Pullerz please feel free to try it out with v1.1.5 and see if you have any more issues on your end.

Pullerz commented 1 year ago

@lhr0909 Working nicely for me now, thanks for fixing this!