Remove unnecessary warning and logging setting code

tatuylonen / wikitextprocessor

Python package for WikiMedia dump processing (Wiktionary, Wikipedia etc). Wikitext parsing, template expansion, Lua module execution. For data extraction, bulk syntax checking, error detection, and offline formatting.

Other

90 stars 23 forks source link

Remove unnecessary warning and logging setting code #270

Closed xxyzz closed 3 months ago

xxyzz commented 3 months ago

DEBUG logging level is only set to our own Logger obejct in wiktextract code and won't affect other packages: tatuylonen/wiktextract#591
I guess the warning filter was added for the "dateparser" package, this was fixed two years ago(dateparser GH issue 1013)

xxyzz commented 3 months ago

Perhaps we should consider use Python's logging library in wikitextprocessor? It could print error messages to error output and save logs to a file so we don't need to worry logs take too much memory. And we also might use other features from logging.

kristian-clausal commented 3 months ago

Let's introduce new imports only when necessary. Wikitextprocessor has worked well without logging up till now, and we have our own error logging system. If there is a pressing need to print logging stuff and no way to do it, then it should be done, but not prematurely.

xxyzz commented 3 months ago

The main problem with our current Wtp.error() methods is they save data in memory not how they are printed, and the recent check JSON code adds more JSON data to the log messages which exacerbates the problem. The server currently has enough RAM but IMO this could be improved.