Add ability to persist logs into a file instead of logging them

atomicptr / crab

🦀 A versatile tool to crawl dozens of URLs from a given source, like a sitemap or an URL list.

MIT License

20 stars 6 forks source link

Add ability to persist logs into a file instead of logging them #4

Closed atomicptr closed 4 months ago

atomicptr commented 4 years ago

$ crab crawl:sitemap https://domain.com/sitemap.xml --output output.log
# NO prints here...

It might probably make sense to have a different option for persisting the logs as a valid json file...

$ crab crawl:sitemap https://domain.com/sitemap.xml --output-json output.json
$ cat output.json
[
    {"status": 200, "url": "https://domain.com/page-1", ...},
    ...
]

atomicptr commented 4 months ago

Closed with #10, #11