Closed Kiro47 closed 4 years ago
Example of new logging with the format [TIME][LOG_LEVEL][CLASS:FUNCTION] MSG
[2020-04-30T03:06:22][INFO][Data_Gathering:info] Acquiring state map [2020-04-30T03:06:23][INFO][Data_Gathering:info] Mapping colleges to states [2020-04-30T03:06:41][INFO][Data_Gathering:info] Getting course data from colleges
Note the FUNCTION
parameter does seem to have some issues due to the wrapper, might have to figure out some magic to get this to work as expected. If you use that formatting with the normal logger directly you'd get the following:
[2020-04-30T03:06:22][INFO][Data_Gathering:get_course_object_list] Acquiring state map [2020-04-30T03:06:23][INFO][Data_Gathering:get_course_object_list] Mapping colleges to states [2020-04-30T03:06:41][INFO][Data_Gathering:get_course_object_list] Getting course data from colleges
Out of scope of PR: we could use asyncio/tornado to send multiple requests at once and significantly speed it up.
That's going to be dealt with in #22
First attempts at refactoring the Banweb scraper into a more modular fashion as we'll quite possibly looking into gathering other data as well soon.
Improvements: