Closed getorca closed 3 years ago
With only LOG_ENABLED = false in settings.py , default behaviour, when running with docker compose you loose logging of unhanded exceptions such as KeyErrors. This is in the crawler service.
KeyError
expected behaviour: unhanded exceptions get logged
It might make sense to have:
LOG_ENABLED = str2bool(os.getenv('LOG_ENABLED', True)) LOG_LEVEL = os.getenv('LOG_LEVEL', 'ERROR')
as the default so that unhandled exceptions are still logged and visible.
This is a Scrapy setting, not a Scrapy Cluster setting. This project has no control over what that setting does or how it impacts scrapy logging.
Closing.
With only LOG_ENABLED = false in settings.py , default behaviour, when running with docker compose you loose logging of unhanded exceptions such as
KeyError
s. This is in the crawler service.expected behaviour: unhanded exceptions get logged
It might make sense to have:
as the default so that unhandled exceptions are still logged and visible.