ImportError: Error loading object 'woaidu_crawler.pipelines.bookfile.WoaiduBookFile': cannot import name MediaPipeline

[lifehack]

我在Ubuntu 12.04中按照说明配置了单mongodb的环境，但是在运行时报错如下：

/woaidu_crawler/woaidu_crawler/spiders/woaidu_detail_spider.py:12: ScrapyDeprecationWarning: woaidu_crawler.spiders.woaidu_detail_spider.WoaiduSpider inherits from deprecated class scrapy.spider.BaseSpider, please inherit from scrapy.spider.Spider. (warning only on first subclass, there may be others) class WoaiduSpider(BaseSpider): /usr/local/lib/python2.7/dist-packages/scrapy/contrib/pipeline/init.py:21: ScrapyDeprecationWarning: ITEM_PIPELINES defined as a list or a set is deprecated, switch to a dict category=ScrapyDeprecationWarning, stacklevel=1) Traceback (most recent call last): File "/usr/local/bin/scrapy", line 4, in execute() File "/usr/local/lib/python2.7/dist-packages/scrapy/cmdline.py", line 143, in execute _run_print_help(parser, _run_command, cmd, args, opts) File "/usr/local/lib/python2.7/dist-packages/scrapy/cmdline.py", line 89, in _run_print_help func(_a, *_kw) File "/usr/local/lib/python2.7/dist-packages/scrapy/cmdline.py", line 150, in _run_command cmd.run(args, opts) File "/usr/local/lib/python2.7/dist-packages/scrapy/commands/crawl.py", line 50, in run self.crawler_process.start() File "/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py", line 92, in start if self.start_crawling(): File "/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py", line 124, in start_crawling return self._start_crawler() is not None File "/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py", line 139, in _start_crawler crawler.configure() File "/usr/local/lib/python2.7/dist-packages/scrapy/crawler.py", line 47, in configure self.engine = ExecutionEngine(self, self._spider_closed) File "/usr/local/lib/python2.7/dist-packages/scrapy/core/engine.py", line 64, in init self.scraper = Scraper(crawler) File "/usr/local/lib/python2.7/dist-packages/scrapy/core/scraper.py", line 66, in init self.itemproc = itemproc_cls.from_crawler(crawler) File "/usr/local/lib/python2.7/dist-packages/scrapy/middleware.py", line 50, in from_crawler return cls.from_settings(crawler.settings, crawler) File "/usr/local/lib/python2.7/dist-packages/scrapy/middleware.py", line 29, in from_settings mwcls = load_object(clspath) File "/usr/local/lib/python2.7/dist-packages/scrapy/utils/misc.py", line 42, in load_object raise ImportError("Error loading object '%s': %s" % (path, e)) ImportError: Error loading object 'woaidu_crawler.pipelines.bookfile.WoaiduBookFile': cannot import name MediaPipeline

scrapy版本为0.22.2，由于不熟悉python以及scrapy这一系列工具，想请教一下如何修改？

补充：去掉 'woaidu_crawler.pipelines.bookfile.WoaiduBookFile', 'woaidu_crawler.pipelines.drop_none_download.DropNoneBookFile', 后可以正常抓取。

[gnemoug]

看了下是scrapy版本问题，你看：https://github.com/scrapy/scrapy/blob/master/scrapy/contrib/pipeline/media.py，它把原来在images中的class移到了media.py，你改下import路径试试。

[imlusion]

求问，到底怎么解决呢？那个链接打不开

[TylerzhangZC]

把distribute_crawler/woaidu_crawler/woaidu_crawler/pipelines/file.py的 line 18:from scrapy.contrib.pipeline.images import MediaPipeline 改成:from scrapy.contrib.pipeline.media import MediaPipeline 然后重新运行就可以了