ViciousPotato
commented
6 years ago Hi could you give me the book id you are trying to download? it seems you gave "install" as book id, it should be in pure digit format
Open gradyhouston opened 6 years ago
ViciousPotato
commented
6 years ago Hi could you give me the book id you are trying to download? it seems you gave "install" as book id, it should be in pure digit format
2017-11-15 22:01:27 [scrapy.utils.log] INFO: Scrapy 1.4.0 started (bot: safaribook) 2017-11-15 22:01:27 [scrapy.utils.log] INFO: Overridden settings: {'NEWSPIDER_MODULE': 'safaribook.spiders', 'SPIDER_MODULES': ['safaribook.spiders'], 'DOWNLOAD_DELAY': 0.25, 'BOT_NAME': 'safaribook'} 2017-11-15 22:01:27 [scrapy.middleware] INFO: Enabled extensions: ['scrapy.extensions.memusage.MemoryUsage', 'scrapy.extensions.logstats.LogStats', 'scrapy.extensions.telnet.TelnetConsole', 'scrapy.extensions.corestats.CoreStats'] 2017-11-15 22:01:27 [scrapy.middleware] INFO: Enabled downloader middlewares: ['scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware', 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware', 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware', 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware', 'scrapy.downloadermiddlewares.retry.RetryMiddleware', 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware', 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware', 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware', 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware', 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware', 'scrapy.downloadermiddlewares.stats.DownloaderStats'] 2017-11-15 22:01:27 [scrapy.middleware] INFO: Enabled spider middlewares: ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware', 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware', 'scrapy.spidermiddlewares.referer.RefererMiddleware', 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware', 'scrapy.spidermiddlewares.depth.DepthMiddleware'] 2017-11-15 22:01:27 [scrapy.middleware] INFO: Enabled item pipelines: [] 2017-11-15 22:01:27 [scrapy.core.engine] INFO: Spider opened 2017-11-15 22:01:27 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min) 2017-11-15 22:01:27 [scrapy.extensions.telnet] DEBUG: Telnet console listening on 127.0.0.1:6023 2017-11-15 22:01:27 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.safaribooksonline.com/> (referer: None) 2017-11-15 22:01:29 [scrapy.downloadermiddlewares.redirect] DEBUG: Redirecting (302) to <GET https://www.safaribooksonline.com/home/> from <POST https://www.safaribooksonline.com/accounts/login/> 2017-11-15 22:01:30 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.safaribooksonline.com/home/> (referer: https://www.safaribooksonline.com/) 2017-11-15 22:01:32 [scrapy.core.engine] DEBUG: Crawled (404) <GET https://www.safaribooksonline.com/nest/epub/toc/?book_id=install> (referer: https://www.safaribooksonline.com/home/) 2017-11-15 22:01:32 [scrapy.spidermiddlewares.httperror] INFO: Ignoring response <404 https://www.safaribooksonline.com/nest/epub/toc/?book_id=install>: HTTP status code is not handled or not allowed 2017-11-15 22:01:32 [scrapy.core.engine] INFO: Closing spider (finished) 2017-11-15 22:01:32 [scrapy.utils.signal] ERROR: Error caught on signal handler: <function close at 0x10415cb18> Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/twisted/internet/defer.py", line 149, in maybeDeferred result = f(*args, *kw) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pydispatch/robustapply.py", line 55, in robustApply return receiver(arguments, **named) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scrapy/spiders/init.py", line 104, in close return closed(reason) File "/Users/grady/Documents/git/safaribooks/safaribook/spiders/safaribooks.py", line 112, in closed shutil.move(self.book_name + '.zip', self.book_title + '-' + self.bookid + '.epub') AttributeError: 'SafariBooksSpider' object has no attribute 'book_title' 2017-11-15 22:01:32 [scrapy.statscollectors] INFO: Dumping Scrapy stats: {'downloader/request_bytes': 2074, 'downloader/request_count': 4, 'downloader/request_method_count/GET': 3, 'downloader/request_method_count/POST': 1, 'downloader/response_bytes': 53803, 'downloader/response_count': 4, 'downloader/response_status_count/200': 2, 'downloader/response_status_count/302': 1, 'downloader/response_status_count/404': 1, 'finish_reason': 'finished', 'finish_time': datetime.datetime(2017, 11, 16, 3, 1, 32, 342177), 'httperror/response_ignored_count': 1, 'httperror/response_ignored_status_count/404': 1, 'log_count/DEBUG': 5, 'log_count/ERROR': 1, 'log_count/INFO': 8, 'memusage/max': 52678656, 'memusage/startup': 52678656, 'request_depth_max': 2, 'response_received_count': 3, 'scheduler/dequeued': 4, 'scheduler/dequeued/memory': 4, 'scheduler/enqueued': 4, 'scheduler/enqueued/memory': 4, 'start_time': datetime.datetime(2017, 11, 16, 3, 1, 27, 332446)} 2017-11-15 22:01:32 [scrapy.core.engine] INFO: Spider closed (finished)