add storage for forms and data [55]

[MartijnR]

using indexedDb

[MartijnR]

This points estimate is a very big guess.

[MartijnR]

• [x] if there is an error opening the database connection (e.g. some schema version conflict) it seems that db.js swallows the error
• [x] add form update mechanism
• [x] store media labels...
• [x] test: cache.get result should have hash property
• [x] store images (KoBo brand)
• [x] safari 7 refuses to store media
• [x] safari 8 also
• [x] IE11 appears to only store the first media file (Blob) for all the media URLs. WTF?
• [x] check mobile browsers
• [x] when loading from storage after restarting Chrome (until 41), the URL respond with 404. Still shown in DB, but not persistent (looks like objectURL is revoked automatically before it can be used)
• [x] check submissions (url /submission/::abcd?)
• [ ] check loading of media labels when repeat with media labels is cloned - WORKS WELL NOW but need to check if this still works after revoking the objectURLS
• [x] getting false update is required events
• [x] update function fails
• [x] test delete form from server -> update returning 404 -> message to user
• [x] test form deactivated -> update returning ? -> message to user
• [x] duplicate resources are still requested in duplicate
• [x] refactor
• [ ] add switch to offline/online view button?
• [x] add config item for 'offline enabled'
• [x] for newly stored form there is a false "update is available" event due to media hash = null (first time) or 'null' (when loaded from cache)

[MartijnR]

https://code.google.com/p/chromium/issues/detail?id=108012

[MartijnR]

confirmed that the issue is still present in beta (39) and dev(40) channel of Chrome. It is not present in Canary ( 41.0.2221.0 ).

Chrome only has blob support for indexeddb since v 37

[MartijnR]

The issue is now also present in Chrome 41.0.2224.3 dev (later version than previous Canary). Wtf? But still not in Canary 41.0.2228.2 canary and 41.0.2229.1 canary. So I guess, Chrome versioning is just confusing for me.

Am carefully optimistic that the Chrome bug with serving media blobs from storage is not present on latest Chrome for Android (Lollipop Dec 5th 2014th). So far so good.

[MartijnR]

Be careful with a solution like localforage that uses multiple storage solutions. A browser update could lead to data loss (e.g. if the preferred indexedDb becomes available).

Also what I don't like about localforage is that you don't get the fancy indexedDb features such as indexing, keyPaths.

[MartijnR]

IE doesn't like blobs to be stored as part of an object. Using simple key, value resolved the issue

[MartijnR]

for Safari, convert Blobs to Base64 and back?? Be prepared for safari adding support in the future, so when retrieving a resource check if it is an instanceof Blob and if not convert to a Blob.

[MartijnR]

arguments against always storing as base64 (dataURI):

• more space required
• if an image occurs multiple times, a URL.createObjectURL is only a reference
• anURL.createObjectURL won't enlarge the DOM?

arguments for:

• no need for special solution needed for Safari
• will work with current buggy Chrome (non-Canary)

[MartijnR]

Database cannot be flushed if another tab is using the same database!

[MartijnR]

Records to do:

• [x] errors thrown in store.setRecord. Error not output to user correctly because of the issues with Foundation reveals. Probably need to switch to different UI lib [:sigh:]
• [x] fix maxSize functionality. May be better to store in surveys table
• [x] various karma tests failing in Firefox
• [ ] store tests failing in Safari
• [ ] refactor store.js further by combining recordStore.update and surveyStore.update ?
• [x] record add does not reject properly if name is duplicate
• [x] show 'Offline forms are experimental. Use only for testing.'
• [x] new modals, the autoClose timer doesn't get cleared (still fires when dialog is closed).
• [x] negative button in dialog.confirm doesn't close dialog any more
• [x] replace foundation buttons
• [x] style buttons in dialogs
• [x] buttons not visible in print dialog (because print stylesheet is active!)
• [x] when print button is in visited state the background is orange
• [x] check renaming functionality
• [x] save media files
• [x] load records XML
• [x] load records with media files
• [x] record counter is per form (dummy)
• [x] finalize submission of queued records (see TODOs in record-queue.js)
• [x] if form is reset, the media labels disappear
• [x] make API return offline forms by default
• [x] add /survey/online endpoint or /survey/offline? (Aggregate will need update for latter)
• [ ] include db.js with bower (my PR was merged)

[MartijnR]

For posterity:

Storing the record count in the properties table (per form with index YYYp:stats ) and not in the surveys table for the following reasons

1. Would like to be able to complete remove a cached form but keep the statistics (e.g. as a tool to work around storage bugs it might be useful to remove a form).
2. Updating a survey recordCounter would otherwise mean retrieving a potentially large survey to read the current counter value and update the whole record with the incremented counter value. Could be a performance issue.
3. The stats could be extended with other interesting survey info (# of successful submissions, instanceIDs of successful submissions, loading performance, errors)