No data returned at endpoint `course/schedule/filter/*/*/*`

[Tigre2325]

When I make a request to https://opendata.concordia.ca/API/v1/course/schedule/filter/*/*/*, no data is returned, while when I add either of the 2 last parameters, ie. */BIOL/*, */BIOL/201 and */*/201, it works perfectly fine.

I know that requesting all the courses takes a long time and a lot of data, but for the web app I am building, I need all the courses (at least all the one from the current session/term, but there is no session parameter).

One solution I found would be to request all the courses at course/catalog and then call course/schedule for each subject, but I feel this is rather cumbersome...

[lib-development-concordiaU]

If you need the whole dataset you can download it from HERE

The issue is probably that the endpoint times out.

Please let me know if this helps,

Cheers

[Tigre2325]

Thank you for your answer! I was hoping to use the API itself, but yes I could work with the raw CSV as well.

I'll try it in the weekend and let you know!

[lib-development-concordiaU]

Ok Good. Glad it helped. I'm closing the issue. You can always open a new one if you need anything from us.

[SpencerMartel]

Will this get resolved? Maybe increasing the time for a time out? Having the data updated daily and fetchable from the API is useful in getting the most current student enrollment numbers.

[lib-development-concordiaU]

I think the amount of data is just too much to be served like this ... What about a time limit ? Say, it will return every from the las 12 month ? or the current semester ? will that work ?

[Tigre2325]

Yes actually that would be awesome! Because I believe @SpencerMartel and I only use the current term, and maybe next one if available. Meaning having some sort of time parameter in the request endpoint, like starting and ending termCode (2222, 2224, etc) so that we could require courses of all terms that are between the 2 requested. Maybe something like course/schedule/filter/*/*/*/2222/2224 or course/schedule/filter/*/*/*?startTermCode=2222&endTermCode=224 would get the courses of 2222, 2223 and 2224 terms (Fall 2022, Fall/Winter 2022/2023, and Winter 2023).

Or I guess just getting the current academic year would be enough with the current endpoint, but if someone later wants to look at older data then it wouldn't be possible....

[SpencerMartel]

I agree that having a term code as a parameter in the request would be a good method of chunking up these large requests. For my needs I use the current semester and any upcoming ones that the service has available. This is generally a massive request but I have no good way to chunk it up as of now.

[lib-development-concordiaU]

Sounds good. We´ll work on implementing this filter

[SpencerMartel]

Fantastic I appreciate the implementation, thank you! Please let us know when it is available so we can update our projects :) If you need any help I am happy to assist where I can

[Tigre2325]

Thank you for trying to implement that!

It will speed up huge time the requests because we will be able to get all the classes of the current term much more quickly than downloading the CSV or requesting courses 1 by 1.

[lib-development-concordiaU]

ok, I implemented a new endpoint

https://opendata.concordia.ca/API/v1/course/scheduleTerm/filter/{subject}/{termCode}

Ex: https://opendata.concordia.ca/API/v1/course/scheduleTerm/filter/BIOL/2242

it is still problematic to retrieve the full semester because there's quite a bit of data and response size might be too big. For example retrieving the whole 2242 semester using postman return a "response is too big error". Maybe other clients can handle it. What it does work well is getting all the schedules for all the courses for a given subject, not sure if that's helpful for you use case.

You can always just download the raw data, parse it and cache it locally, maybe that's the most reliable way to do it.

Cheers.

[Tigre2325]

Ok thank you for the implementation! I'll try it most probably this weekend and see how I can make it work for my use.

[SpencerMartel]

I appreciate the solve! Going to use this new implementation

[SpencerMartel]

Just testing this out now and it doesn't seem to be working. This call should return all BIOL classes for Winter 2023 but it is returning all BIOL classes since 2014. Here is the first item returned from that call.

{
"courseID": "002625",
"termCode": "2141",
"session": "13W",
"subject": "BIOL",
"catalog": "201",
"section": "50",
"componentCode": "LEC",
"componentDescription": "Lecture",
"classNumber": "1161",
"classAssociation": "1",
"courseTitle": "INTRODUCTORY BIOLOGY",
"topicID": "",
"topicDescription": "",
"classStatus": "Active",
"locationCode": "LOY",
"instructionModeCode": "P",
"instructionModeDescription": "In Person",
"meetingPatternNumber": "1",
"roomCode": "CC320",
"buildingCode": "CC",
"room": "320",
"classStartTime": "18.30.00",
"classEndTime": "21.00.00",
"modays": "N",
"tuesdays": "Y",
"wednesdays": "N",
"thursdays": "N",
"fridays": "N",
"saturdays": "N",
"sundays": "N",
"classStartDate": "07/05/2014",
"classEndDate": "19/08/2014",
"career": "Undergraduate",
"departmentCode": "BIOLOGY",
"departmentDescription": "Biology",
"facultyCode": "AS",
"facultyDescription": "Faculty of Arts & Science",
"enrollmentCapacity": "72",
"currentEnrollment": "66",
"waitlistCapacity": "0",
"currentWaitlistTotal": "0",
"hasSeatReserved": ""
}

[volovikariel]

ok, I implemented a new endpoint

https://opendata.concordia.ca/API/v1/course/scheduleTerm/filter/{subject}/{termCode}

Ex: https://opendata.concordia.ca/API/v1/course/scheduleTerm/filter/BIOL/2242

it is still problematic to retrieve the full semester because there's quite a bit of data and response size might be too big. For example retrieving the whole 2242 semester using postman return a "response is too big error". Maybe other clients can handle it. What it does work well is getting all the schedules for all the courses for a given subject, not sure if that's helpful for you use case.

You can always just download the raw data, parse it and cache it locally, maybe that's the most reliable way to do it.

Cheers.

Considering we're able to retrieve all of the courses without too many issues already (it works fine for me, hearing back from the API calls takes a while, but that's regardless of the endpoint), I'm surprised that retrieving all of the courses between start and end termCodes fails :confused: . @lib-development-concordiaU

[volovikariel]

Just testing this out now and it doesn't seem to be working. This call should return all BIOL classes for Winter 2023 but it is returning all BIOL classes since 2014. Here is the first item returned from that call.

{
"courseID": "002625",
"termCode": "2141",
"session": "13W",
"subject": "BIOL",
"catalog": "201",
"section": "50",
"componentCode": "LEC",
"componentDescription": "Lecture",
"classNumber": "1161",
"classAssociation": "1",
"courseTitle": "INTRODUCTORY BIOLOGY",
"topicID": "",
"topicDescription": "",
"classStatus": "Active",
"locationCode": "LOY",
"instructionModeCode": "P",
"instructionModeDescription": "In Person",
"meetingPatternNumber": "1",
"roomCode": "CC320",
"buildingCode": "CC",
"room": "320",
"classStartTime": "18.30.00",
"classEndTime": "21.00.00",
"modays": "N",
"tuesdays": "Y",
"wednesdays": "N",
"thursdays": "N",
"fridays": "N",
"saturdays": "N",
"sundays": "N",
"classStartDate": "07/05/2014",
"classEndDate": "19/08/2014",
"career": "Undergraduate",
"departmentCode": "BIOLOGY",
"departmentDescription": "Biology",
"facultyCode": "AS",
"facultyDescription": "Faculty of Arts & Science",
"enrollmentCapacity": "72",
"currentEnrollment": "66",
"waitlistCapacity": "0",
"currentWaitlistTotal": "0",
"hasSeatReserved": ""
}

This endpoint returns the same values (notice the termCode): https://opendata.concordia.ca/API/v1/course/scheduleTerm/filter/BIOL/2141.

I suspect there's no filtering being done at all right now, so it's returning the response equivalent to https://opendata.concordia.ca/API/v1/course/schedule/filter/*/BIOL/* @SpencerMartel

[SpencerMartel]

I suspect there's no filtering being done at all right now, so it's returning the response equivalent to https://opendata.concordia.ca/API/v1/course/schedule/filter/*/BIOL/* @SpencerMartel

This is my assumption too, I believe if it was filtering we would be able to access all courses in a semester using course/schedule/filter/*/{termcode} without the API timing out. Wanted to bring it to @lib-development-concordiaU's attention.

[Tigre2325]

To add onto @SpencerMartel, I also tested the new endpoint and the termCode filter does not work...

I think I'll go with the raw CSV data for now, even though it means downloading a 40 MB file, until the endpoint is working properly as I would like to have my MVP working as soon as possible.

[lib-development-concordiaU]

Sorry Guys,

There was an issue moving the changes to the production server. You may try again the new endpoint, I believe it should be working now.

EX: https://opendata.concordia.ca/API/v1/course/scheduleTerm/filter/BIOL/2224

[Tigre2325]

I just tested the endpoint quickly on my phone and I can confirm it seems to work: I called it with parameters BIOL/2224 and correctly got all the BIOL classes for Winter 2023. I even tried the wildcard parameter */2224 to get all courses of Winter 2023, and I got them fairly quickly.

I'll integrate it with my app and let you know, but this quick test is very promising, so thank you!

[SpencerMartel]

Fantastic it seems to be filtering well, and responds in a reasonable time when requesting all classes for a given semester.

I appreciate the solve, this works well for my implementation!

[volovikariel]

Would it be too much trouble to implement a range instead? Meaning /course/scheduleTerm/filter/BIOL/2222/2224.

It would still time out if we passed in the start and end terms as being years apart (just like it is now), but it would allow us to make a simple API call to get, say, a year's worth of data without any issue I would imagine.

Thoughts? @Tigre2325 @SpencerMartel

[SpencerMartel]

I think chunking up responses by year is reasonable to maintain stable responses. I think allowing a range would be nice but you won't know the max number of semesters before you run into a time out.

For my use case it doesn't cause much trouble to do it with this current implementation.