Closed qasim closed 8 years ago
This looks really good so far.
It looks like we're gonna end up having very similar entries (just with differing section_id
& section_code
) if an instructor is teaching the same course multiple times. Is this something we should consider revising?
Also, just to clarify, we aren't planning to keep data for sections that have no book requirements, right? It might be worth it, but the dataset would be filled with a lot of duplicate content from the Course Finder scrapers.
@kshvmdn I'm thinking the data will be book first instead of section. So it'll be a book schema, and the book has ISBN, price, author, etc. and whether it is required by any courses or not. Then people can query for required books by course to have the textbooks returned to them.
In terms of the duplicate entries, I definitely need to add something that will merge all same books and then have the list of sections / courses that it corresponds to inside the schema. I'll post a version 1 of what the schema could look like within the week.
Alright, it now outputs textbooks. Now what's left is to merge unique textbooks and consolidate the courses/meeting sections.
There we go. Should be good for another look over.
This isn't done, but would like some eyes on it.