search

trailsroc / map-data

Map data for the #TrailsRoc app
https://trailsroc.org/trailsroc-the-app
Creative Commons Zero v1.0 Universal
1 stars 0 forks source link

Latest map additions aren't selectable on iOS #7

Closed mmertsock closed 4 years ago

mmertsock commented 4 years ago

Fairy trail and the Whiting Road pink trail are visible on the map, but in the iOS app you can't search for or select them. Works fine on the web app.

The iOS app uses the Mapbox Style for rendering and looking up identifiers of selected features, but in the background it downloads the associated Dataset and uses its raw geojson for everything that needs feature metadata: selection panels, search/browse screens, etc. I don't see these new trails in the dataset, only in the style and tileset. So that's likely the problem.

The web app gets all the metadata it needs directly from the Mapbox style layers, which is why it's working correctly.

mcbeth commented 4 years ago

I think I've broken everything, and have no idea how we're really supposed to update datasets. You're going to have to intervene and tell me what I was supposed to do.

mmertsock commented 4 years ago

Yeah no worries. The app is still fully functional for now, just can’t see the info when tapping specifically on those new trails. I’m happy to get the dataset in sync, just didn’t want to step on toes if you were in the middle of updating or had perhaps updated the dataset somewhere else and just needed to copy over. And going forward we’ll have to ensure the process is documented. To start with, I’ll document what I did to update the dataset for this issue.

mcbeth commented 4 years ago

To build the dataset, I'm just running make_geosjon.rb followed by geojson-merge, but I've been uploading that to the tileset directly (since you can do a replace there) rather than messing with the dataset. I'll be fascinated to discover what the right thing to do is.

mcbeth commented 4 years ago

I think things might be desperately broken for iOS. I just got a messages saying "it kept giving me the "Updating Map..." message. no luck. iPhone X SW v. 13.5

mcbeth commented 4 years ago

Yup, it looks like the datasetID changed, and so the iOS app is now broken. (Line 15 of MapboxConfig.swift?). When you're back around, we need to have a chat about how I broke everything so bad and how to fix it.

mmertsock commented 4 years ago

Interesting, not sure how the dataset ID could have changed. But it is hard-coded into the iOS app so we will need to either somehow restore a dataset with the expected ID (not likely possible) or update the iOS code. Existing installs of the iOS app work fine, from what I can see. It caches the dataset pretty aggressively so I guess it's robust to this scenario.

I have some features I've implemented but not yet released, so was planning to do an iOS release soon anyway, and can include the updated dataset ID if needed.

Also, in the iOS app I'll look into whether it's possible to derive the dataset ID from the current Mapbox style that's being rendered; or possibly use the tileset instead of dataset, so that there's one less hard-coded identifier in the app.

mmertsock commented 4 years ago

I never figured out a really clean way to update datasets. Last I checked, and from what I can tell this is still true, the Mapbox API doesn't have an endpoint that allows you to replace the entire geojson contents of a dataset, you can only insert/update/delete individual features. So we could possibly write a script to update the entire contents of a dataset with the current data, but it would be non-trivial.

mcbeth commented 4 years ago

I'll work on the dataset update in a safe spot.

From the looks of the API and iOS. It looks like we could agree on a name of a dataset, like "release", or "prod-v1-build1", and then you could send two queries, one to get the list of datasets and the second to get the dataset whose name matches. That doesn't really reduce your hardcoding, but gives us an identifier we can control rather than mapbox controlling it.

It looks like the tileset API has a "recipe" item, so you could theoretically grab the style (ugh), look for the sources, grab the tileset source recipes, and work back to the dataset that way.

mmertsock commented 4 years ago

Here's my plan:

  1. Configure the app to look up a dataset by name (e.g. "prod-v1-build1") instead of ID
  2. GET /datasets API call to list the datasets
  3. Choose the dataset with specified name and newest modification date (in case multiple exist with the same name)
  4. Cache the ID of the selected dataset so later data fetches in the app can directly retrieve the dataset. If the API indicates the dataset with that ID is not found, go back to step 2

Could configure the app to match dataset names using a regex instead of exact string match if we want to make things more flexible, though I'm not sure that's necessary — any major changes would likely need an app update for compatibility anyway.

mcbeth commented 4 years ago

That sounds eminently reasonable. I'd rather a name match rather than regex for now. As you say, major changes would force app updates, and we could revisit then

mcbeth commented 4 years ago

Yeah, the API sucks. The two options I see are to slowly (40/minute, or 75 minutes) delete all features, then add them back (craptacular)

Or, I could request the whole dataset, do some of the pattern matching I did before to estimate the ids, and then add what's missing, update what's changed. Not awesome, as it will break. Many times.

Or, we delete the whole dataset, create a new one with the same name, and upload again. Which breaks the tileset, as those are no longer linked, so that gets deleted/created. Which breaks the style, so we update that.

Did you do it by hand before using the geojson directory for wholesale adds, and in their GUI editor for anything else?

mcbeth commented 4 years ago

Is there a way to get what you need from the map API for features selections? Like what the web map does, but less sucky?

I'm so far behind on feature parity on the new Android app.

mmertsock commented 4 years ago

Previously I’ve updated the dataset manually. Really small changes I would just hand-edit. Larger changes, I would delete all the contents for a specific park from the dataset (easy to do actually, by dragging a selection rectangle over all the features in a park’s area), and then upload the updated geojson file for that park.

But going forward, if the iOS app (and presumably Android app) fetches datasets by name instead of ID, it would be totally fine to delete and rebuild datasets when we need to bulk update data. Much simpler.

I don’t see a reasonable way to get all the map data needed for search/browse functionality from the map style or tileset. The native (and JavaScript) SDKs for the map view and map style are focused on user interactions with specific features, so you can only fetch visible/selected features on demand. No good way to fetch everything including data that isn’t currently visible. Which is why instead the iOS app just downloads and caches the entire dataset. This is also why the web app doesn’t have search/browse capability but you can see info about selected features: it can get what it needs from the style object for selected features, but full search/browse requires the dataset download, which is possible to implement but I just hadn’t got around to doing so.

mmertsock commented 4 years ago

I have dataset lookup by name working in the iOS app. It’s configured with dataset name prod-v1-build1. Privacy policy link is integrated also. Doing some final testing today then I’ll submit the update to the App Store.