Sorry this is going be a huge pull request

[Davidster]

resolves #32

Summary

Changes made to the sequence scraper/sequence JSON spec:

1. Removed course name and course credits to decrease redundancy in DB
2. Added new detail to JSON spec allowing for choice of a list of courses (OR)
3. Assorted fixes to the sequence scraper logic

The code changes here are heavily hand-tested so you can be sure they work properly.

1. Removed course name/course credits from JSON spec

Now the course objects in the sequence JSON files will only contain the course code. Before, we also had the course name and the number of credits.

note: as a result of this change, we will need to update the mongosequences sevlet (SequenceProvider.java) by having it fill up the missing data by grabbing it from the course data DB

Sample of old spec

"courseList": [
    {
        "code": "COMP 232",
        "name": "Mathematics for Computer Science",
        "isElective": "false",
        "electiveType": "",
        "credits": "3"
    },
    {
        "code": "COMP 248",
        "name": "Object-Oriented Programming I",
        "isElective": "false",
        "electiveType": "",
        "credits": "3"
    },
    {
        "code": "ENGR 201",
        "name": "Professional Practice & Responsibility",
        "isElective": "false",
        "electiveType": "",
        "credits": "1.5"
    },
    {
        "code": "ENGR 213",
        "name": "Applied Ordinary Differential Equations",
        "isElective": "false",
        "electiveType": "",
        "credits": "3"
    },
    {
        "code": "",
        "name": "",
        "isElective": "true",
        "electiveType": "Science",
        "credits": ""
    }
]

Sample of new spec

"courseList": [
    {
        "code": "COMP 232",
        "isElective": "false",
        "electiveType": ""
    },
    {
        "code": "COMP 248",
        "isElective": "false",
        "electiveType": ""
    },
    {
        "code": "ENGR 201",
        "isElective": "false",
        "electiveType": ""
    },
    {
        "code": "ENGR 213",
        "isElective": "false",
        "electiveType": ""
    },
    {
        "code": "",
        "isElective": "true",
        "electiveType": "Science"
    }
]

2. Added ability to list a choice of courses

As described in issue #32, sometimes instead of one single course, a list of courses that you can choose from are put into the reccomended sequences. My proposed solution (which is implemented in this PR) is to simply put these courses into an array where you would normally expect to see a course object. It is implied that one of the courses in this array is to be picked, so the user will be able to choose one of them in the frontend app.

Example from SOEN-Games-Coop

The above recommended sequence portion results in the following json data:

{
    "season": "fall",
    "courseList": [
        {
            "code": "SOEN 490",
            "isElective": "false",
            "electiveType": ""
        },
        {
            "code": "ENGR 301",
            "isElective": "false",
            "electiveType": ""
        },
        {
            "code": "SOEN 321",
            "isElective": "false",
            "electiveType": ""
        },
        {
            "code": "COMP 376",
            "isElective": "false",
            "electiveType": ""
        },
        [
            {
                "code": "COMP 477",
                "isElective": "false",
                "electiveType": ""
            },
            {
                "code": "COMP 345",
                "isElective": "false",
                "electiveType": ""
            },
            {
                "code": "COMP 472",
                "isElective": "false",
                "electiveType": ""
            },
            {
                "code": "",
                "isElective": "true",
                "electiveType": "Program"
            }
        ],
        {
            "code": "COMP 345",
            "isElective": "false",
            "electiveType": ""
        },
        {
            "code": "COMP 472",
            "isElective": "false",
            "electiveType": ""
        }
    ],
    "isWorkTerm": "false"
},
{
    "season": "winter",
    "courseList": [
        {
            "code": "SOEN 385",
            "isElective": "false",
            "electiveType": ""
        },
        {
            "code": "ENGR 392",
            "isElective": "false",
            "electiveType": ""
        },
        {
            "code": "SOEN 490",
            "isElective": "false",
            "electiveType": ""
        },
        [
            {
                "code": "COMP 476",
                "isElective": "false",
                "electiveType": ""
            },
            {
                "code": "COMP 345",
                "isElective": "false",
                "electiveType": ""
            },
            {
                "code": "COMP 472",
                "isElective": "false",
                "electiveType": ""
            },
            {
                "code": "",
                "isElective": "true",
                "electiveType": "Program"
            }
        ],
        {
            "code": "COMP 345",
            "isElective": "false",
            "electiveType": ""
        },
        {
            "code": "COMP 472",
            "isElective": "false",
            "electiveType": ""
        }
    ],
    "isWorkTerm": "false"
}

3. Assorted fixes to the sequence scraper logic

There are also some situations where the course code will be followed by a descriptor. Some examples:

COMP 248*, COMP 248+, COMP 248 (alternative).

All of these cases will now end up being recorded as COMP 248, fully ignoring the descriptor. I made this judgement call on my own as it seemed that accounting for these exceptional situations would be far from worth our time. If you want to look into it further, be my guest...

I also fixed certain cases where empty courses would be put into the sequence and also where work terms would not be properly added.

Final words

I spent several hours going through every sequence JSON file one by one and comparing it directly to its source website.

The sequence json spec is now ALMOST perfect, with the exception of one final issue #39 which I will ask your guys' opinion on since I'm not sure how to deal with it.

[PeterGhimself]

@Davidster message