Memory usage concerns

[tomsaleeba]

We have a user who is having memory usage issues.

Device: Samsung SM-N960F OS: Chrome Mobile 86.0.4240 Version: 0b0f626.20201014-130445

The app has 17 queued observations, each with at least 3 photos but probably more. Most of the photos are high resolution so the total size of all the local records is quite large. It's gotten to the point where trying to do a sync or create a new obs causes the "aw, snap" message as the browser has run out of memory.

We tried to open an incognito window and load WOW there, to start with a clean slate. We tried to upload one observation, but we couldn't even save a single observation with 3 (9mb) photos. The error message we were seeing was:

Failed to read large IndexedDB value

Perhaps incognito windows place a tighter limit on how much space IndexedDB can use.

The memory usage of that incognito tab was ~250mb after trying to save the observation (originally with 6 photos). This doesn't make sense because:

1. there are no local records taking up space
2. the remote photos are using the low res versions and we only render a single thumbnail photo

I'm curious to know how we were still eating up so much memory with no local records.

This screenshot is from the main app:

These screenshots are from the incognito window:

It's interesting in the bottom of that last screenshot that we see __sentry_original_ mentioned. We did trigger a few Sentry errors. Is Sentry causing a memory leak by holding references?

[tomsaleeba]

Part of this problem was solved as part of #69 . Now this device can load the obs list and it managed to upload most of the observations. There were 3 that would still cause the tab to crash as it ran out of memory. These three had the most photos, more than 5, attached. The cause is that we still load all the photos, so we can serialise them and send a complete bundle to the SW.

This approach won't work. The rationale was to have a clear boundary between the SW and the app code, but that separation was broken when we had the SW set the outcome on records.

The new approach:

• avoid caching what's in IndexedDB where possible, just iterate the records instead. It's pretty fast.
• don't give the SW its own DB. Instead, the wow-obs and wow-photos DB(s) are the source of truth. The app and the SW both run off those records
• introduce some sort of hash/ETag on the records so we can tell when they've been changed
• consider removing the "blocked action" concept and instead just clobber the record. Updating the ETag (as above) will indicate it's been touched, then we know it needs to be re-processed.
• only ever load one photo at a time. So for the processing pipeline:
  • load a photo
  • make a request
  • handle the response
  • onto the next photo