apify / crawlee-python

Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
https://crawlee.dev/python/
Apache License 2.0
33 stars 1 forks source link

Polish up logging #157

Open janbuchar opened 1 month ago

janbuchar commented 1 month ago

In the current state, we make a new logger in every module that needs to log something. There is CrawleeLogFormatter, which handles logging in the console.

vdusek commented 1 month ago

Also, a question that comes to mind: do we want to override the default Python logger formatting? considering Crawlee is a library

janbuchar commented 1 month ago

Also, a question that comes to mind: do we want to override the default Python logger formatting? considering Crawlee is a library

And also a framework... It is not unusual for frameworks to define their own loggers. But if a client wants to override that, we should make it easy.