DadosAbertosDeFeira / tcm-ba

Raspagem de dados do TCM-BA 📄
https://tcm.ba.gov.br
MIT License
6 stars 0 forks source link

Permitir filtragem por unidade no raspador de documentos #29

Closed anapaulagomes closed 3 years ago

anapaulagomes commented 3 years ago

Atualmente recebemos o argumento porém ao tentar escolher uma unidade recebemos o erro:

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/site-packages/scrapy/utils/defer.py", line 120, in iter_errback
    yield next(it)
  File "/usr/local/lib/python3.8/site-packages/scrapy/utils/python.py", line 353, in __next__
    return next(self.data)
  File "/usr/local/lib/python3.8/site-packages/scrapy/utils/python.py", line 353, in __next__
    return next(self.data)
  File "/usr/local/lib/python3.8/site-packages/scrapy/core/spidermw.py", line 56, in _evaluate_iterable
    for r in iterable:
  File "/usr/local/lib/python3.8/site-packages/scrapy/spidermiddlewares/offsite.py", line 29, in process_spider_output
    for x in result:
  File "/usr/local/lib/python3.8/site-packages/scrapy/core/spidermw.py", line 56, in _evaluate_iterable
    for r in iterable:
  File "/usr/local/lib/python3.8/site-packages/scrapy/spidermiddlewares/referer.py", line 342, in <genexpr>
    return (_set_referer(r) for r in result or ())
  File "/usr/local/lib/python3.8/site-packages/scrapy/core/spidermw.py", line 56, in _evaluate_iterable
    for r in iterable:
  File "/usr/local/lib/python3.8/site-packages/scrapy/spidermiddlewares/urllength.py", line 40, in <genexpr>
    return (r for r in result or () if _filter(r))
  File "/usr/local/lib/python3.8/site-packages/scrapy/core/spidermw.py", line 56, in _evaluate_iterable
    for r in iterable:
  File "/usr/local/lib/python3.8/site-packages/scrapy/spidermiddlewares/depth.py", line 58, in <genexpr>
    return (r for r in result or () if _filter(r))
  File "/usr/local/lib/python3.8/site-packages/scrapy/core/spidermw.py", line 56, in _evaluate_iterable
    for r in iterable:
  File "/usr/local/lib/python3.8/site-packages/tcmba/spiders/consulta_publica.py", line 184, in get_search_results
    unit_payload = unit_payloads.pop(0)
IndexError: pop from empty list