search

cobalt-uoft / uoft-scrapers

Public web scraping scripts for the University of Toronto.
https://pypi.python.org/pypi/uoftscrapers
MIT License
48 stars 14 forks source link

Add UTSC / UTM exam scrapers #44

Closed kashav closed 8 years ago

kashav commented 8 years ago

Forced to hardcode some stuff (eg. month, year) and there doesn't seem to be any alternative. I think it's just something we'll have to update each exam period.

https://github.com/cobalt-uoft/uoft-scrapers/issues/35

qasim commented 8 years ago

Do you think it's possible for us to map the exam periods to dates, and then have a method that can determine the period from that?

Maybe the following mapping:

Exam period Date range (inclusive)
DEC15 2015-12-01 to 2015-12-31
APR16 2016-04-01 to 2016-04-30
JUNE16 2016-06-01 to 2016-06-30
AUG16 2016-08-01 to 2016-08-31

Of course the ranges are a littler generous, but as long as there is no overlap we will support some potential edge cases if an exam has a weird time?

arkon commented 8 years ago

Yeah, I feel like it should be possible to generate the period based on the year/month of the exams.

kashav commented 8 years ago

It's definitely possible, but what happens when the date doesn't fall between any of those ranges? I think we'd still be forced to use a hardcoded date (correct me if I'm wrong, but I assumed you're suggesting to use this with today's date).

Also, when are schedules usually posted for June/August? I don't think they'd be up the whole month (I'm not sure though).

qasim commented 8 years ago

You'd be comparing with each of the exam's dates, which should produce the correct results unless there is some anomaly with a course.

For June/August, if they are up, the dates for both exams would fall in June or August for each. The huge range is to account for the +- ~1 week of when the semesters start. It is definitely generous but shouldn't hurt to have so big.

arkon commented 8 years ago

Wouldn't it be based on when the exams actually are? If an exam is on 2016-04-22 for example, I would assume it's in the APR16 period.

Unless I'm not understanding what the problem is?

kashav commented 8 years ago

My bad, I misunderstood what you meant.

Made changes to find month, year, period based on exam date. https://github.com/cobalt-uoft/uoft-scrapers/pull/44/commits/eb5f3a37f6431a4751f7d696571eb2686a9c26f7

qasim commented 8 years ago

No worries! Looks great. I will test this and merge ASAP.

kashav commented 8 years ago

Woops #movefastandbreakthings

https://github.com/cobalt-uoft/uoft-scrapers/pull/44/commits/0d94bcc83df8ce6199bb237ebc23503ac5addb1c