search

robatwilliams / openai-excel-functions

Create OpenAI chat completions from Excel formulas
ISC License
29 stars 3 forks source link

LRU cache #6

Open chrisprice opened 9 months ago

chrisprice commented 9 months ago

Calling the API can be expensive and time consuming. In many cases Excel deals with only calling it when necessary but there's also a long list of cases when it calls it unnecessarily (Volatile Actions: Actions that trigger calculation).

To mitigate this an LRU cache could be used (with appropriate concessions for non determinism, temperature, etc.) which makes use of the Storage API. It may also make sense to make use of the browser Compression Streams API.

robatwilliams commented 9 months ago

Notably, our functions are recalculated at workbook open - despite them not being marked as volatile in the metadata.