Maintenance mode details

[andygup]

Thanks to all who have asked questions about this library going into maintenance mode notice. I wanted to consolidate and share this information for others:

• Now that ArcGIS JS API v4.0 has been released, I'll continue to answer open issues and fix major bugs in offline-editor-js v3.x.
• There won't be any enhancements added to offline-editor-js v3.x.
• If you are using ArcGIS JS API v3.15+ then the library will work.
• This library won't work with ArcGIS JS API v4.0.
• I don't have any immediate plans to migrate this project to ArcGIS JS API 4.x, however if someone in the community is interested in working on that task please feel free to ping me.
• We continue to encourage everyone to look at the suite of native Runtime SDKs for their officially integrated and significantly more robust offline capabilities.

[stdavis]

Hey, @andygup. Sorry to see that this project is not being actively developed anymore. Do you know why ESRI isn't putting resources behind offline web? Is it that the tech isn't ready yet? Or that they just want to gain more traction with the runtime products?

[andygup]

Hey there @stdavis, yep the focus for offline mapping is on Runtimes, Qt (AppStudio) and Xamarin.

Just to mention a few of the technical reasons: those platforms/languages have significantly more robust and better defined mechanisms for gracefully handling the mechanics of being fully offline, they also work in many more use cases, and the native APIs are much better at working with larger local datasets than using Browser-based APIs.

If you wanna chat about it more, feel free to shoot me an email or DM me on Twitter.

[davinci0428]

@andygup, I've been investigating the possibility of creating browser-based GIS apps that use Google's new Progressive Web App technology to enable offline use. Can you tell me if the PWA technology overcomes some of the robustness issues that otherwise exists in browser-based GIS apps? If you're not acquainted with PWA technology, can you at least give me a better idea of what those limitation are that caused you no longer develop this project?

[andygup]

@davinci0428 thanks for the question! I get asked this question quite a bit, so here are some of my thoughts. There are strong technical reasons for using our Runtime SDKs for offline.

Can you tell me if the PWA technology overcomes some of the robustness issues that otherwise exists in browser-based GIS apps?

A PWA can overcome 'some' of the robustness issues found in this library, but still can't come close to handling all the use cases that our Runtime SDKs handle with ease. A few of the challenges include cross-browser support, data storage limitations and offline/online detection.

Cross-browser support is challenging with PWAs for example service workers are currently not supported on IE 11, Edge, Safari or iOS Safari -- reference.

Browsers are not designed to handle large amounts (500MB+) of data storage I/O consistently as well as iOS and Android native SDKs, and are significantly more memory constrained. I mention large amounts of data because most offline mapping applications require basemaps, and offline basemap tile packages can get quite large and often larger than 500MB especially if you want to be able to zoom in to ground level using imagery (e.g. Zoom Level 23). Here's a good article - Offline Storage for Progressive Web Apps

And, offline/online detection is still challenging, for example Google suggests using the Network Information API in the PWA checklist, but it is experimental -- native online/offline detection is significantly better.

[davinci0428]

@andygup, thanks for the quick reply! You provided just the information I needed to better weigh the pros and cons of using your offline editor in the field. I think what you've created is a great tool that can be used in the field under limited conditions, but I've decided that, as you've said, it's probably not robust enough for my needs.

[andygup]

Many thanks for all the on-going questions about browser data storage limitations for JavaScript. Here is a summary of best practices and considerations.

For web mapping applications, I still recommend a maximum of 100MB (Megabytes) of storage even for PWAs (Progressive Web Apps). There are blog posts and articles stating that you can use more storage. That may be true for many use cases, however for mapping application there a variety of mitigating factors that can severely limit this amount:

• Cross browser requirements. Browser vendors are not consistent in how they implement storage and manage memory under-the-hood. YMMV.
• Large storage amounts combined with heavy memory I/O and heavy read/write requirements that are inherent in mapping apps are a very difficult thing to manage with JavaScript. Even when properly managed it can cause slowdowns, lockups and crashes.
• JavaScript offers no explicit capability for managing memory or garbage collection. Overall browser memory footprint can grow very large. There is a greater than 1:1 ratio between the amount of storage you put into the browser and the memory and on-disk storage the browser allocates to manage it. A 10MB file, for example, could theoretically result in the browser using 50MBs , or greater, of storage and it varies by browser types.
• The combined memory usage across all browser tabs can lead to memory instability. There's no control over other browser tabs and the amount of memory and storage they are using.
• It's not easy (impossible?) to get users to limit the number of tabs, regularly delete the browser cache and restart the browser.

In summary, 100MB maximum browser storage is a best practice. It's very challenging to manage browser storage well. The greater amount of storage that you use, expect slowdowns, lockups and crashes especially in memory constrained devices such as smartphones and tablets.

[khanguslee]

If anyone is looking to get some sort of offline map caching support using arcGIS 4.x with your PWA, have a look at https://github.com/Esri/nearby-javascript. Idea is to use workbox to setup your service worker to cache the arcGIS map tiles.

[JonForest]

PWAs continue to push offline storage limits - from https://web.dev/storage-for-the-web/#how-much it seems that 1Gb is a reasonably safe bet. However, I wonder if with service workers this becomes a relatively easy problem?

If we use a service worker to cache map tiles locally then presumably we pay the API cost associated with that tile layer fetch? How does the new pricing structure work if users are then consuming the map tiles from the cache? I couldn't find anything specifically in the terms of use (https://www.esri.com/content/dam/esrisites/en-us/media/legal/product-specific-terms-of-use/e300.pdf) about this.

[andygup]

it seems that 1Gb is a reasonably safe bet.

Based on experience, it's the heavy I/O requirements of a browser-based mapping app that caused browser crashes and lockups in the past. Heavy I/O can cause CPU/GPU bottlenecking, memory thrashing and memory leaks.

This is why we only recommend using ArcGIS Runtime or ArcGIS Pro for full offline use cases. They are designed for full offline and aren't dependent on the browser.

How does the new pricing structure work if users are then consuming the map tiles from the cache?

You'd need to talk to your account representative about pricing and licensing. Make sure you are in compliance with the ArcGIS Master Agreement 3.2(c) Permitted Uses:

Customer may take ArcGIS Online basemaps offline through Esri Content Packages and subsequently deliver (transfer) them to any device for use with licensed ArcGIS Runtime applications and ArcGIS Desktop. Customer may not otherwise scrape, download, or store Data.

[glenndejaeger]

Hi all,

We are creating mobile apps based on Ionic and Capacitor. The single JS based code base uses the Arcgis JS API. Currently we build a web app and an Android app from the source code. In the future we will also build an iOS app and a desktop app (via Electron).

Our web app and Android app support offline:

• We use a sqlite/sql.js db to keep a local copy of the data
• Data changes are sent to the server via the server API and updated in the local db
• A queue of all the changes is stored locally
• If an API call succeed, the call is removed from the queue.
• If an API call fails because the user is offline, the call is re-executed from the queue when the user is gets back online

Currently we don't have a service worker which means that our static assets (JS, CSS, HTML, images, fonts, esri map tiles, ...) are not cached for offline use. That's not an issue because our main use case is the mobile app. Offline is supported in the mobile app because all static assets are 'baked' into the app itself.

We're now looking for a solution to have esri map functionality offline:

• offline availability of the tiles that were fetched when you were checking the map while you were offline
• offline availability of tiles that were not explicitly fetched => tiles with parts of the map from the neighborhood. A smart prefetching algorithm knows which tiles to fetch
• offline functionality: draw on the map and persist these changes when the user gets back online

We have servers that expose .vtpk and .mmpk files, including custom layers. Can these be consumed via the JS api? Can we fetch specific tiles (eg tiles showing part of a map at a specific geolocation)?

[andygup]

Hi @glenndejaeger the JS API does not support tile packages or offline workflows, but the native Runtime SDKs offer full support, for example: https://developers.arcgis.com/android/api-reference/reference/com/esri/arcgisruntime/mapping/MobileMapPackage.html.