search

mnich0ls / evee-sd

An event aggregation app for events in San Diego
1 stars 2 forks source link

Web Scraper: Meetup #12

Closed mnich0ls closed 5 years ago

mnich0ls commented 5 years ago

Scrape events from: https://www.meetup.com/find/events/?allMeetups=true&radius=50&userFreeform=San+Diego%2C+CA&mcId=z92101&mcName=San+Diego%2C+CA&eventFilter=all

Events should be persisted to the scraped_events collection (see Database) with the following metadata:

source_url: https://www.meetup.com date_scraped: (the ISO 8601 timestamp for the time the event was pulled from the source site) status: new

See Event Attributes for the event attributes that should be scraped.

jremi commented 5 years ago

@mnich0ls please assign me to this task.

I just performed a preliminary initial integration into Meetups API. They have an endpoint resource that will provide what we need. There is no reason to webscrape this page (unless you really want that method)... since we can get the data directly from there API. The endpoint is api.meetup.com/find/upcoming_events

Please take a look at this JSON response I just got for the San Diego area. https://gist.github.com/jremi/6d0ae568790b5338e7034aebcd76dd9e

mnich0ls commented 5 years ago

Perfect. Please use the API instead.

mnich0ls commented 5 years ago

Whatever makes any task easier or faster is fair game to me.