search

rmax / dirbot-mysql

Scrapy project based on dirbot to show how to use Twisted's adbapi to store the scraped data in MySQL.
117 stars 55 forks source link

ImportError: Error loading object 'googledir.pipelines.FilterWordsPipeline': No module named exceptions #3

Closed laapsaap closed 11 years ago

laapsaap commented 11 years ago

Hi, its not working anymore.

Traceback (most recent call last): File "./scrapy-ctl.py", line 7, in execute() File "/usr/lib/python2.6/site-packages/scrapy/cmdline.py", line 143, in execute _run_print_help(parser, _run_command, cmd, args, opts) File "/usr/lib/python2.6/site-packages/scrapy/cmdline.py", line 88, in _run_print_help func(_a, *_kw) File "/usr/lib/python2.6/site-packages/scrapy/cmdline.py", line 150, in _run_command cmd.run(args, opts) File "/usr/lib/python2.6/site-packages/scrapy/commands/crawl.py", line 46, in run spider = self.crawler.spiders.create(spname, opts.spargs) File "/usr/lib/python2.6/site-packages/scrapy/command.py", line 34, in crawler self._crawler.configure() File "/usr/lib/python2.6/site-packages/scrapy/crawler.py", line 44, in configure self.engine = ExecutionEngine(self, self._spider_closed) File "/usr/lib/python2.6/site-packages/scrapy/core/engine.py", line 63, in init self.scraper = Scraper(crawler) File "/usr/lib/python2.6/site-packages/scrapy/core/scraper.py", line 66, in init** self.itemproc = itemproc_cls.from_crawler(crawler) File "/usr/lib/python2.6/site-packages/scrapy/middleware.py", line 50, in from_crawler return cls.from_settings(crawler.settings, crawler) File "/usr/lib/python2.6/site-packages/scrapy/middleware.py", line 29, in from_settings mwcls = load_object(clspath) File "/usr/lib/python2.6/site-packages/scrapy/utils/misc.py", line 40, in load_object raise ImportError, "Error loading object '%s': %s" % (path, e) ImportError: Error loading object 'googledir.pipelines.FilterWordsPipeline': No module named exceptions

laapsaap commented 11 years ago

My python and lib installation was messed up.

Diaspar4u commented 10 years ago

what did you do to fix it?? i'm getting an exact same error...

rmax commented 10 years ago

@Diaspar4u it's been a long time since I did check this project and didn't noticed that Google's directory website has been shutdown. At the other hand, there has been a change in the import from scrapy.core.exceptions to scrapy.exceptions, but now it doesn't make sense to update this project to make it work with latest Scrapy code.

You can still use the pipelines.py code removing the unneeded imports for the mysql pipeline.