ingress: data sources and pipeline

[ear7h]

Where and how are we gathering data?

Are we interfacing with APIs, if so what do we need to do to get them?

• a nice API to work with would be google calendar

Are we going to scrape webpages?

• what will we use to scrape webpages?

[ear7h]

One data source would be google calendar which exposes CalDAV endpoints https://developers.google.com/calendar/caldav/v2/guide

[ear7h]

Looks like most of the orgs use facebook and we can scrape it or use an api https://github.com/catcatio/fb-event-scraper https://developers.facebook.com/docs/graph-api/reference/v8.0/group/events