Make scraper for Course API - Githubissues

ScottyLabs / course-api-v2

📃 A REST API for retrieving CMU course information and faculty evaluation data

8 stars 0 forks source link

Make scraper for Course API #6

Open gramliu opened 4 years ago

gramliu commented 4 years ago

Make scraper to remove dependence on course-api v1

hdavidethan commented 4 years ago

This will require looking at two data sources:

Course: https://enr-apps.as.cmu.edu/open/SOC/SOCServlet/courseDetails?COURSE=15213&SEMESTER=F20
FCE: https://wwwp3.smartevals.com/Reporting/Students/Wizard.aspx?Type=Classes*
- Might have to export because it's blocked behind CMU Login

Some experimentation is needed on parsers (shouldn't be too complicated):

HTML parsers for Course:
1. https://www.npmjs.com/package/x-ray
2. https://www.npmjs.com/package/jssoup (similar to BeautifulSoup for Python)
CSV parsers for FCE:
1. https://www.npmjs.com/package/papaparse

hdavidethan commented 3 years ago

Add cross-listed courses is now part of this issue. Ref #11