search

eliangcs / pystock-crawler

(UNMAINTAINED) Crawl and parse financial reports (XBRL) from SEC EDGAR, and daily stock prices from Yahoo Finance
MIT License
311 stars 105 forks source link

Merge failed if result is empty #16

Closed eliangcs closed 9 years ago

eliangcs commented 9 years ago

pystock-crawler reports command raises error when merging subfiles if one of them is empty.

Error output:

2014-10-08 11:20:48-0400 [scrapy] INFO: Merging files to /home/eliang/2014/10/01/reports.csv
2014-10-08 11:20:48-0400 [-] ERROR: Traceback (most recent call last):
2014-10-08 11:20:48-0400 [-] ERROR:   File "/home/eliang/.virtualenvs/pystock/bin/pystock-crawler", line 264, in <module>
2014-10-08 11:20:48-0400 [-] ERROR:     main()
2014-10-08 11:20:48-0400 [-] ERROR:   File "/home/eliang/.virtualenvs/pystock/bin/pystock-crawler", line 250, in main
2014-10-08 11:20:48-0400 [-] ERROR:     crawl(spider, symbols, start_date, end_date, output, log_file, batch_size)
2014-10-08 11:20:48-0400 [-] ERROR:   File "/home/eliang/.virtualenvs/pystock/bin/pystock-crawler", line 153, in crawl
2014-10-08 11:20:48-0400 [-] ERROR:     merge_files(output, output_files, ignore_header=True)
2014-10-08 11:20:48-0400 [-] ERROR:   File "/home/eliang/.virtualenvs/pystock/bin/pystock-crawler", line 104, in merge_files
2014-10-08 11:20:48-0400 [-] ERROR:     f.next()  # Ignore CSV header
2014-10-08 11:20:48-0400 [-] ERROR:   File "/home/eliang/.virtualenvs/pystock/lib/python2.7/codecs.py", line 681, in next
2014-10-08 11:20:48-0400 [-] ERROR:     return self.reader.next()
2014-10-08 11:20:48-0400 [-] ERROR:   File "/home/eliang/.virtualenvs/pystock/lib/python2.7/codecs.py", line 615, in next
2014-10-08 11:20:48-0400 [-] ERROR:     raise StopIteration
2014-10-08 11:20:48-0400 [-] ERROR: StopIteration