Shrink Storage Footprint

[dermeck]

Right now the extension stores the whole state of the feedSlice into Local Storage. This is not ideal a might lead to problems when more feeds are added.

Most of the data is fetched on startup. How should items be handled that were in the feed before and are missing now (is persistence needed)? It may also useful to clear parts of the store depending on certain criteria (by age https://github.com/dermeck/ytrss/issues/13, only keep the latest x items per feed, only keep and store favorite items). It may alsobe useful to transform the data before storing (e.g. shorten the keys of json properties and/or use hash instead of full url).

Storage Limits: https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/storage/local

[dermeck]

dummy data for ~3000 feed items written to a file with JSON.stringfy produce a ~700KB file

[dermeck]

Maybe items should not be stored at all and instead only be fetched. (only store "isRead" information). This way the data might fit into the sync storeage (later)

isRead Information:

• id of "oldest" unread item (all older iotems are imölicitely read) + ids of other unread items
• rely on order (from API) or is a date always present?
• use full model internally and only shrink stored (= keep isRead flag?) info or alway work with shrunken info

https://developer.mozilla.org/en-US/docs/Mozilla/Add-ons/WebExtensions/API/storage/sync#storage_quotas_for_sync_data

[dermeck]

Tests: 4595 Items: full export: 1.2 MB (1,177,901 bytes) minified keys: 1.1 MB (1,055,015 bytes) (=> -122886Bytes) ~10% only feeds (no folders) + shrinked keys 1.0 MB (1,045,038 bytes) (=> -9977 Byte) - depends on folders..

https://github.com/dermeck/feeds-sidebar/commit/d621da06b7fb776f12d444675294117760492e54

only feedUrls (144) 9.7 kB (9,712 bytes) https://github.com/dermeck/feeds-sidebar/commit/9df8e4349f40ff29ddbbd7177b81e4f9e8374287

[dermeck]

It makes sense to keep title etc in local storage so that we don't have to wait for feed loading on startup. In the future there might be a mix of local and sync data (feed urls + isRead data). Prepare data structure for improved isRead Tracking.

TODO Adapt handling for isRead-Tracking

• use hashCode function to create ids from url for feed items
• store url and isRead Data (readUntil (all read until lastModified date) + isRead[] or isUnread[] with ids)
• this might break existing data => ensure import with folders works properly

Branch: https://github.com/dermeck/feeds-sidebar/tree/shrink-storage