search

eliangcs / pystock-crawler

(UNMAINTAINED) Crawl and parse financial reports (XBRL) from SEC EDGAR, and daily stock prices from Yahoo Finance
MIT License
311 stars 105 forks source link

IndexError on PLT report #8

Closed eliangcs closed 10 years ago

eliangcs commented 10 years ago

Command pystock-crawler reports PLT -o output.csv raised this error:

2014-08-10 20:41:16+0800 [edgar] ERROR: Spider error processing <GET http://www.sec.gov/Archives/edgar/data/914025/000091402513000049/plt-20130630.xml>
    Traceback (most recent call last):
      File "/Users/eliang/.virtualenvs/pystock-crawler/lib/python2.7/site-packages/twisted/internet/base.py", line 824, in runUntilCurrent
        call.func(*call.args, **call.kw)
      File "/Users/eliang/.virtualenvs/pystock-crawler/lib/python2.7/site-packages/twisted/internet/task.py", line 638, in _tick
        taskObj._oneWorkUnit()
      File "/Users/eliang/.virtualenvs/pystock-crawler/lib/python2.7/site-packages/twisted/internet/task.py", line 484, in _oneWorkUnit
        result = next(self._iterator)
      File "/Users/eliang/.virtualenvs/pystock-crawler/lib/python2.7/site-packages/scrapy/utils/defer.py", line 57, in <genexpr>
        work = (callable(elem, *args, **named) for elem in iterable)
    --- <exception caught here> ---
      File "/Users/eliang/.virtualenvs/pystock-crawler/lib/python2.7/site-packages/scrapy/utils/defer.py", line 96, in iter_errback
        yield next(it)
      File "/Users/eliang/.virtualenvs/pystock-crawler/lib/python2.7/site-packages/scrapy/contrib/spidermiddleware/offsite.py", line 26, in process_spider_output
        for x in result:
      File "/Users/eliang/.virtualenvs/pystock-crawler/lib/python2.7/site-packages/scrapy/contrib/spidermiddleware/referer.py", line 22, in <genexpr>
        return (_set_referer(r) for r in result or ())
      File "/Users/eliang/.virtualenvs/pystock-crawler/lib/python2.7/site-packages/scrapy/contrib/spidermiddleware/urllength.py", line 33, in <genexpr>
        return (r for r in result or () if _filter(r))
      File "/Users/eliang/.virtualenvs/pystock-crawler/lib/python2.7/site-packages/scrapy/contrib/spidermiddleware/depth.py", line 50, in <genexpr>
        return (r for r in result or () if _filter(r))
      File "/Users/eliang/.virtualenvs/pystock-crawler/lib/python2.7/site-packages/scrapy/contrib/spiders/crawl.py", line 67, in _parse_response
        cb_res = callback(response, **cb_kwargs) or ()
      File "/Users/eliang/Projects/pystock-crawler/pystock_crawler/spiders/edgar.py", line 56, in parse_10qk
        loader = ReportItemLoader(response=response)
      File "/Users/eliang/Projects/pystock-crawler/pystock_crawler/loaders.py", line 473, in __init__
        self.add_xpath('amend', '//dei:AmendmentFlag')
      File "/Users/eliang/Projects/pystock-crawler/pystock_crawler/loaders.py", line 364, in add_xpath
        self.add_value(field_name, values, *processors, **kw)
      File "/Users/eliang/.virtualenvs/pystock-crawler/lib/python2.7/site-packages/scrapy/contrib/loader/__init__.py", line 45, in add_value
        self._add_value(field_name, value)
      File "/Users/eliang/.virtualenvs/pystock-crawler/lib/python2.7/site-packages/scrapy/contrib/loader/__init__.py", line 59, in _add_value
        processed_value = self._process_input_value(field_name, value)
      File "/Users/eliang/.virtualenvs/pystock-crawler/lib/python2.7/site-packages/scrapy/contrib/loader/__init__.py", line 117, in _process_input_value
        return proc(value)
      File "/Users/eliang/.virtualenvs/pystock-crawler/lib/python2.7/site-packages/scrapy/contrib/loader/processor.py", line 27, in __call__
        next_values += arg_to_iter(func(v))
      File "/Users/eliang/Projects/pystock-crawler/pystock_crawler/loaders.py", line 63, in __call__
        return value.xpath('./text()')[0].extract()
    exceptions.IndexError: list index out of range