search

ufosc / course-scheduler

A dynamic course scheduler for planning out college semesters.
MIT License
9 stars 5 forks source link

Identify endpoints / Pages for data sources #2

Open envp opened 7 years ago

envp commented 7 years ago

We need to identify API endpoints for various pieces of information to completely build the dependencies for all of the courses.

Status:

Purpose Endpoint Docs/Wiki Comments
Schedule https://one.uf.edu/apix/soc/schedule/?term= $TERM &category= $CATEGORY &lastrow=$LASTROW TODO There are more parameters check the wiki for info.
Description https://one.uf.edu/apix/soc/cdesc/ $COURSE_CODE TODO This is the only query parameter, however we may need to build a crude parser for the response containing pre-reqs/coreqs.
Dependencies TODO TODO Course description offers some rough unstructured info, but can we do better?
Evaluations TODO TODO Evaluate using the evaluations page for this info

Assignments:

Dependencies - @pikachuster

Updates

Schedule - Scrapy project Dependencies / Descriptions - Currently being added to the scrapy based code. Evaluations - Probably going to use scrapy / selenium here whichever comes through first reliably

ghost commented 7 years ago

Regarding evaluations, probably should start looking at https://evaluations.ufl.edu/results/. Tried looking at the page source of a single evaluation, seems some Javascript magic is going on usually towards the end? CRTL+F for "api" didn't seem to bring up anything relevant, except that they seem to be using JQuery.

Saw something interesting, <script id="evaluationTemplate" type="text/x-jquery-tmpl"> on line 165 of my sample page, https://evaluations.ufl.edu/results/instructor.aspx?ik=-957811086. Also of interest is <script type="text/javascript"> on line 276 and on...

Those two scripts, combined, seem to be pulling data from somewhere and displaying it on screen.

ghost commented 7 years ago

Using Firefox Inspector on a single table cell, I'm able to see the raw data between the table tags. But just where and how is the data even coming in?

envp commented 7 years ago

Nice find! It seems to be a jQuery template, which gets filled using the JSON of the response. I need to compare that with the response json fully to be sure if that is all it is doing, since it sends only a single request to the API.

Will edit with confirmation