nliautaud / trakttvstats

A chrome extension adding various improvements to trakt.tv
MIT License
29 stars 5 forks source link

TMDb API caching #28

Open nliautaud opened 6 years ago

nliautaud commented 6 years ago

Cache API responses in LocalStorage to retrieve already known data.

nliautaud commented 6 years ago

2017-10-02 18_26_13-extensions

2017-10-02 18_28_51-petzeroun s tv show and movie watched history - trakt tv

Can't find performance tests about chrome.storage.local.get, so tests may be needed to determine the best method between those two possibilities :

gervasiocaj commented 6 years ago

I wrote some performance tests, available at https://gist.github.com/gervasiocaj/b8c326f01cbcddad2a9f9cc8b080c823.

The sample data is structured in the following format: http://www.filltext.com/?rows=2&id={string|8}&username={username}&password={string|5}&text={lorem|30}&pretty=true

// myData format:
// [{ "id": "04Kwnn2Q", "username": "DChandrasekahran", ... }]

// myDataAsObject format:
// { "04Kwnn2Q": { "username": "DChandrasekahran", ... } }

// myDataIds format:
// [ "04Kwnn2Q" ]

The code run was the following:

const storageGet = data => new Promise(resolve => chrome.storage.local.get(data, resolve))
const storageSet = data => new Promise(resolve => chrome.storage.local.set(data, resolve))

console.time('storageSet test all data')
await storageSet({ data: myData })
console.timeEnd('storageSet test all data')

console.time('storageGet test all data')
await storageGet('data')
console.timeEnd('storageGet test all data')

// -----------------------------------------

console.time('storageSet test each data')
await storageSet(myDataAsObject)
console.timeEnd('storageSet test each data')

console.time('storageGet test each data')
await storageGet(myDataIds)
console.timeEnd('storageGet test each data')

The dataset length for the following results is 1000.

storageSet test all data: 70.7080078125ms
storageGet test all data: 28.281005859375ms
storageSet test each data: 62.2861328125ms
storageGet test each data: 24.7099609375ms

I am open to suggestions for broader tests, if needed.

nliautaud commented 6 years ago

Amazing !

So there is a measurable performance hit, but very small, between getting everything and picking some. If timings increase is moderate with a huge cache object (10.000, 100.000 ?), it would confirm that it's largely preferable to load everything once.

What was the ids count of myDataIds ?