ulixee / secret-agent

The web scraper that's nearly impossible to block - now called @ulixee/hero
https://secretagent.dev
MIT License
667 stars 44 forks source link

Debug log implementation pollutes other packages' output #366

Open claabs opened 2 years ago

claabs commented 2 years ago

Many npm packages that provide debug logging do so with a package like debug. In the case of debug, packages namespace their logs so you can pinpoint where debug logs are output from.

Unfortunately, the way secret-agent uses the DEBUG environment variable, it will output some overkill logs even when I namespaced to another package (e.g. DEBUG=my-package).

It would be nice if the log output wasn't just a truthy on the DEBUG, but instead was namespaced (e.g. DEBUG=secret-agent or DEBUG=secret-agent:error). For bonus points, it could even look for its namespace in an advanced value like DEBUG=connect:bodyParser,connect:compress,connect:session,secret-agent

blakebyrnes commented 2 years ago

@claabs great points. When we ported SecretAgent out of Ulixee, we were plugging in a "pino" logger via the commons logger, and we left this bare-bones logger in place that is definitely putting out too much by default.

You can also inject a logger that uses the debug module if you want to. There's no public documentation right now, but you could inject a logger (https://github.com/ulixee/secret-agent/blob/a4f681e0e74c55cca2c7024c9beed148f7737af4/commons/Logger.ts#L191) looking at the code that assigned debug targets to the modules (each log entry passes in the code module).

We'll make that variable more meaningful for the default case. That sounds like something I've been wanting lately as well