search

disinfoRG / ZeroScraper

Web scraper made by 0archive.
https://0archive.tw
MIT License
10 stars 2 forks source link

sql connection error while using ns.py update #67

Closed andreawwenyi closed 4 years ago

andreawwenyi commented 4 years ago 
2020-02-16 17:11:30 [twisted] CRITICAL: Unhandled error in Deferred:

Traceback (most recent call last):
  File "/Users/wyw/.local/share/virtualenvs/NewsScraping-I6zyEuYv/lib/python3.7/site-packages/scrapy/crawler.py", line 184, in crawl
    return self._crawl(crawler, *args, **kwargs)
  File "/Users/wyw/.local/share/virtualenvs/NewsScraping-I6zyEuYv/lib/python3.7/site-packages/scrapy/crawler.py", line 188, in _crawl
    d = crawler.crawl(*args, **kwargs)
  File "/Users/wyw/.local/share/virtualenvs/NewsScraping-I6zyEuYv/lib/python3.7/site-packages/twisted/internet/defer.py", line 1613, in unwindGenerator
    return _cancellableInlineCallbacks(gen)
  File "/Users/wyw/.local/share/virtualenvs/NewsScraping-I6zyEuYv/lib/python3.7/site-packages/twisted/internet/defer.py", line 1529, in _cancellableInlineCallbacks
    _inlineCallbacks(None, g, status)
--- <exception caught here> ---
  File "/Users/wyw/.local/share/virtualenvs/NewsScraping-I6zyEuYv/lib/python3.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "/Users/wyw/.local/share/virtualenvs/NewsScraping-I6zyEuYv/lib/python3.7/site-packages/scrapy/crawler.py", line 85, in crawl
    self.spider = self._create_spider(*args, **kwargs)
  File "/Users/wyw/.local/share/virtualenvs/NewsScraping-I6zyEuYv/lib/python3.7/site-packages/scrapy/crawler.py", line 108, in _create_spider
    return self.spidercls.from_crawler(self, *args, **kwargs)
  File "/Users/wyw/.local/share/virtualenvs/NewsScraping-I6zyEuYv/lib/python3.7/site-packages/scrapy/spiders/__init__.py", line 50, in from_crawler
    spider = cls(*args, **kwargs)
  File "/Users/wyw/Codes/NewsScraping/newsSpiders/spiders/update_dcard_spider.py", line 25, in __init__
    site_id=site_id, current_time=int_current_time
  File "/Users/wyw/.local/share/virtualenvs/NewsScraping-I6zyEuYv/lib/python3.7/site-packages/pugsql/statement.py", line 154, in __call__
    r = self._module._execute(self._text, *multiparams, **params)
  File "/Users/wyw/.local/share/virtualenvs/NewsScraping-I6zyEuYv/lib/python3.7/site-packages/pugsql/compiler.py", line 116, in _execute
    raise NoConnectionError()
pugsql.exceptions.NoConnectionError: No connection engine is configured. Pass a connection string to the module's connect method, or pass a SQLAlchemy engine to the set_engine method.

2020-02-16 17:11:30 [twisted] CRITICAL: 
Traceback (most recent call last):
  File "/Users/wyw/.local/share/virtualenvs/NewsScraping-I6zyEuYv/lib/python3.7/site-packages/twisted/internet/defer.py", line 1418, in _inlineCallbacks
    result = g.send(result)
  File "/Users/wyw/.local/share/virtualenvs/NewsScraping-I6zyEuYv/lib/python3.7/site-packages/scrapy/crawler.py", line 85, in crawl
    self.spider = self._create_spider(*args, **kwargs)
  File "/Users/wyw/.local/share/virtualenvs/NewsScraping-I6zyEuYv/lib/python3.7/site-packages/scrapy/crawler.py", line 108, in _create_spider
    return self.spidercls.from_crawler(self, *args, **kwargs)
  File "/Users/wyw/.local/share/virtualenvs/NewsScraping-I6zyEuYv/lib/python3.7/site-packages/scrapy/spiders/__init__.py", line 50, in from_crawler
    spider = cls(*args, **kwargs)
  File "/Users/wyw/Codes/NewsScraping/newsSpiders/spiders/update_dcard_spider.py", line 25, in __init__
    site_id=site_id, current_time=int_current_time
  File "/Users/wyw/.local/share/virtualenvs/NewsScraping-I6zyEuYv/lib/python3.7/site-packages/pugsql/statement.py", line 154, in __call__
    r = self._module._execute(self._text, *multiparams, **params)
  File "/Users/wyw/.local/share/virtualenvs/NewsScraping-I6zyEuYv/lib/python3.7/site-packages/pugsql/compiler.py", line 116, in _execute
    raise NoConnectionError()
pugsql.exceptions.NoConnectionError: No connection engine is configured. Pass a connection string to the module's connect method, or pass a SQLAlchemy engine to the set_engine method.
andreawwenyi commented 4 years ago

fix by commit 8afe4df