search

urbanarchive / qpl-name-explorer

White label site for QPL Name Explorer
2 stars 2 forks source link

ETL From "Monuments" Airtable to Github ("Flat" approach) #2

Open allthesignals opened 2 years ago

allthesignals commented 2 years ago

Embed QPL’s submission form from Airtable - documentation here (dedicated page)

Discovery/investigation: will Github "flat" approach work? Build a Github Action with a CRON job to run regular pulls of data from Airtable. Questions: how will images work? how many records will this be? can this be split up into a REST-like structure? (Shallow index of records, deep view of single records, optional shallow GeoJSON representation of shallow index).

Airtable Database https://airtable.com/appS1fzGsI76K6IPO/tbliHXBoSwurWIqNH/viw2rOai7HkLtlbbB?blocks=hide

allthesignals commented 2 years ago

Discovery

  1. Environment variables for GH actions — docs?
  2. Which table(s) to use?
  3. How often to run? Nightly?
  4. Images! Seems we are in luck. Direct embed with thumbnails: image
allthesignals commented 2 years ago

This has been setup and resolved as far as I can tell. Only follow-up questions I have are:

  1. Which table(s) to use?

Using this one for now!

allthesignals commented 2 years ago

See simple rendered geojson from etl

allthesignals commented 2 years ago

Need to make it recusive so it pulls all data

allthesignals commented 2 years ago

https://github.com/tulsa-policy-institute/masthead/blob/master/etl.js

allthesignals commented 2 years ago