How to tackle local storage limit?

[jakubrohleder]

I recently learned quite an interesting fact I've never thought about - the limit of local storage is quite low (approx. 5200000B~=5MB for chrome), exceeding this limit can produce weird results and might be a problem for bigger apps.

I have two ideas but non of it seems to be perfect:

• compressing data - I tried compressing data with LZW compression, but out of the box it significantly slowed the performance of the package. Each time the model is fetched its key, as well as its relationship keys are dumped to localstore (and compressed). The one solution I can find is to dump cached data to localstore, not during each request but in some time intervals and during $rootScope destroy event. It requires some work so I prefer to think it through before writing the code.
• storing newest data only - some data can be removed from the locastore using cache algorithm or something similar. Question is: which algorithm to choose to provide the best user experience?

Do you find it a high priority issue? Can you find any other solutions?

[eduardmartinez]

I think it should be good to have a TTL option for clearing data storage which could be set at storage definition. I have seen it in another libraries like angular-cache. It allows to select which type of storage I want to use (localStorage or sessionStorage), if you consider it relevant. :D

[jakubrohleder]

@eduardmartinez TTL is good idea, I thought about it during development, but somehow have forgotten about it later. I'll add this idea to the roadmap.

Nevertheless it does not solve a problem - you still can overload localstore with relatively new data. Imagine scrolling facebook page down for few minutes - amount of posts/comments/likes etc. data would be huge and can exceed 5MB limit.

[LavaToaster]

I think realistically, this isn't an issue that can be worked around easily and will more likely fall upon the developer to decide what should and shouldn't be cached. Which some may not even bother as it brings in complexity of dealing with out of date data. (No evidence that may be the case) The only use case I have for storing a json object in localStorage is for the initial user payload for an angular app.

FWIW, There is IndexedDB that is available, it looks as if that has a much higher quota. Some of the browser support is not quite there yet, but IMO the apps that should be storing a-lot of data on the client for caching would be offline apps. Just food for thought. (There is also localForage that can be used to fall back to localStorage if a browser doesn't support IDB)

[ryantemple]

There is also https://github.com/ozantunca/locally which has TTL support and compression

[jakubrohleder]

@Lavoaster you are totally right! I got so much focused on thinking how to resolve the issue, that I have not analysed if it really exists. As you pointed localstorage is used only to provide the initial payload and in this 5MB is more the enough to populate user screen.

In the future I'd love to add support for fully or semi offline apps, but then indexedDB (with approx. 2GB limit) is the answer.

@ryantemple I'd like to keep external dependency to the minimum, so using extra package just to support TTL is not necessary and I'd rather remove outdated objects, not the whole store key. Moreover I tried LZW compression, but in the current data flow it made the app to slow down considerably.

Thanks for all the tips, according to @Lavoaster advice I'll add an annotation to the readme file, so each developer can deal with it by himself.