search

calpoly-csai / cal-poly-knowledge-graph

3 stars 9 forks source link

Club Scraper #1

Open Waidhoferj opened 2 years ago

Waidhoferj commented 2 years ago

Create a new scraper in the scrapers folder that gets info on Cal Poly Campus clubs. See the Club object in models.py to understand the schema. Review CollegeScraper as an example of a web scraper template.

Braden50 commented 2 years ago

Working in "club-scraper" branch. Cal poly now uses an API that we can utilize. In attempting to see if I need to be signed in to see officers, the cal poly portal login stopped working. Will have to check in later

Braden50 commented 2 years ago

Looks like to see officer or member information, you have to be signed in. This complicates things because it means our requests now need to be made in a consistent session or cookies need to be passed around.