scrapy / scrapy

Scrapy, a fast high-level web crawling & scraping framework for Python.
https://scrapy.org
BSD 3-Clause "New" or "Revised" License
51.16k stars 10.35k forks source link

Spider.logger not logging custom extra information #6323

Closed bloodforcream closed 5 days ago

bloodforcream commented 3 weeks ago

I noticed the implicit behavior of the Spider.logger: when logging with extra, extra ultimately do not end up in the log because they are overwritten by default process method of LoggerAdapter

Current logic:

>>> self.logger.info("test log", extra={"test": "very important information"})
{"message": "test log", "spider": "spider_name"}

Expected logic:

>>> self.logger.info("test log", extra={"test": "very important information"})
{"message": "test log", "spider": "spider_name", "test": "very important information"}