icssc / peterportal-api-next

API that provides easy access to public data from UC Irvine. Developed for Anteaters, by Anteaters.
https://docs.icssc.club/anteaterapi
MIT License
6 stars 0 forks source link

feat: ✨ add calendar scraper and dump endpoint #132

Closed ecxyzzy closed 9 months ago

ecxyzzy commented 9 months ago

Summary

Issues

Closes #111; closes #124.

ecxyzzy commented 9 months ago

Yeah I narrowed down the issue, it's just a data integrity problem that I need to resolve as I may have imported faulty data into the dev database. I'll do that at some point and re-request your review then.

Update for posterity: It was, in fact, not "just a data integrity problem"; the algorithm was inherently ill-suited to handle the myriad edge cases with the formatting of the UCI Registrar's Quarterly Academic Calendar, so I rewrote the calendar library using a more sane algorithm that locates the entries based on the keywords, rather than assuming that their locations relative to the table's first entry will remain constant.

This should also make it more extensible should we desire additional fields down the line.