Add 4th layer to the Moped AGOL resource to house exploded multipoint geometry as points

[frankhereford]

Associated issues

https://github.com/cityofaustin/atd-data-tech/issues/17999

Testing

• Get on this branch and replicate the production DB to your machine
• Make yourself a env_file or ask me to wormhole you one.
  • Thanks to @mddilley for observing that those with existing env files may have to update this line as follows:

    HASURA_ENDPOINT=http://host.docker.internal:8080/v1/graphql

• cd moped-etl/arcgis/
• docker compose run arcgis;
  • This will run with the default argument, a sole -d, which means update the existing data with changes from the last seven days.
• docker compose run arcgis -d 2024-08-24T18:33:51.026046+00:00
  • This will do a similar data update, handling changes since the supplied datetime.
• docker compose run arcgis -f
  • This is a full replace. This is painful, as it's more likely than not that you'll encounter an AGOL error, on their end. When this happens, you just have to try again. And you should try again until it works so the data is restored to the correct state. It took me maybe 10 tries over the course of half an hour to finally get through a full replace, but it did do it eventually.
  • Update on Sept 10: I just ran this doing a full replace and got it first try, so YMMV for the better. 🤞
• Review the data on AGOL: https://austin.maps.arcgis.com/home/item.html?id=1c084c8756a84e6db7e2796c98c850a2

Thanks

Thanks to @mddilley for his help on this one -- brainstorming and rubber ducking. 🙏 🦆 🙏

Also thanks to @johnclary for his input and request to stick with our tried and true patterns. 💯

---

Ship list

• [x] Code reviewed
• [x] Product manager approved
• [x] Product manager checked DB view dependencies ~- [ ] Product manager added to QA test script if applicable~

[frankhereford]

@johnclary thanks for all your feedback! I am going to incorporate it as I rework this to use a view based on Mike and your direction here. I left you a note above with some more details about the choices and where it's going -- thanks again!

[johnclary]

ah—i see the separate PR now wrt the README. thanks!

[frankhereford]

@johnclary wrote:

it's my own (bad?) habit of expecting my docker container to have access to it's siblings. if we're going to remove this, would you also update the env_template with the correct local Hasura endpoint?

I updated environment template file -- thank you for that. 5b74d4a5311b54bef9852db45f49220cb7c923ac.

To your observation about using network=host or not; I guess my tendency is to try to lean into docker's ability to encapsulate and containerize, which makes me prefer a container not having access to the host's other services and being able to open ports on the host without explicit permission. Maybe this is a reflection of why I'm prone to write docker-compose.yaml files; i'm all about cementing that sibling relationship in configuration files, where you can have docker make one container genuinely dependent on another being available and all that. Your habit isn't any more wrong than mine; I think they are just different.

[frankhereford]

@johnclary wrote:

We have the option of slimming down the new exploded geometry view

Absolutely - great call. The view, query, and ETL have been refactored to do this.

[mddilley]

@frankhereford i merged the base branch (md-18877-update-proj-dev-status) into main but did not delete so i didn't pull the branch out from under you and if you prefer to manually re-target main