eliangcs / pystock-crawler

(UNMAINTAINED) Crawl and parse financial reports (XBRL) from SEC EDGAR, and daily stock prices from Yahoo Finance
MIT License
311 stars 105 forks source link

Add stats to log #12

Closed eliangcs closed 9 years ago

eliangcs commented 10 years ago

It is difficult to find out the root cause of #2. To make it easier to debug, it'd be better if we collect some stats in the log, such as number of start URLs that were fired and the number of requests that were throttled.