Closed nith-ch closed 1 month ago
This is the Scrapy issue tracker, please ask questions about your code in suitable places.
Hi. Do you mean to give me to ask in the "Bug report" room?
You don't have a bug report, that's the point. Do you mean you selected "Question / Help" when submitting this and then deleted all the text without reading it?
Hi all,
I want to get data from the auction website for my project but I tried many times it still shows Crawled 0 pages error. I am not sure something is wrong with my code. Please advise me.
My code is below:
My error is here
2024-03-21 10:39:56 [scrapy.middleware] INFO: Enabled extensions: ['scrapy.extensions.corestats.CoreStats', 'scrapy.extensions.telnet.TelnetConsole', 'scrapy.extensions.logstats.LogStats'] 2024-03-21 10:39:56 [scrapy.crawler] INFO: Overridden settings: {'BOT_NAME': 'au_SQL', 'FEED_EXPORT_ENCODING': 'utf-8', 'NEWSPIDER_MODULE': 'au_SQL.spiders', 'REQUEST_FINGERPRINTER_IMPLEMENTATION': '2.7', 'ROBOTSTXT_OBEY': True, 'SPIDER_MODULES': ['au_SQL.spiders'], 'TWISTED_REACTOR': 'twisted.internet.asyncioreactor.AsyncioSelectorReactor'} 2024-03-21 10:39:56 [scrapy.middleware] INFO: Enabled downloader middlewares: ['scrapy.downloadermiddlewares.robotstxt.RobotsTxtMiddleware', 'scrapy.downloadermiddlewares.httpauth.HttpAuthMiddleware', 'scrapy.downloadermiddlewares.downloadtimeout.DownloadTimeoutMiddleware', 'scrapy.downloadermiddlewares.defaultheaders.DefaultHeadersMiddleware', 'scrapy.downloadermiddlewares.useragent.UserAgentMiddleware', 'scrapy.downloadermiddlewares.retry.RetryMiddleware', 'scrapy.downloadermiddlewares.redirect.MetaRefreshMiddleware', 'scrapy.downloadermiddlewares.httpcompression.HttpCompressionMiddleware', 'scrapy.downloadermiddlewares.redirect.RedirectMiddleware', 'scrapy.downloadermiddlewares.cookies.CookiesMiddleware', 'scrapy.downloadermiddlewares.httpproxy.HttpProxyMiddleware', 'scrapy.downloadermiddlewares.stats.DownloaderStats'] 2024-03-21 10:39:56 [scrapy.middleware] INFO: Enabled spider middlewares: ['scrapy.spidermiddlewares.httperror.HttpErrorMiddleware', 'scrapy.spidermiddlewares.offsite.OffsiteMiddleware', 'scrapy.spidermiddlewares.referer.RefererMiddleware', 'scrapy.spidermiddlewares.urllength.UrlLengthMiddleware', 'scrapy.spidermiddlewares.depth.DepthMiddleware'] 2024-03-21 10:39:56 [scrapy.middleware] INFO: Enabled item pipelines: ['auSQL.pipelines.SQLlitePipeline'] 2024-03-21 10:39:56 [scrapy.core.engine] INFO: Spider opened 2024-03-21 10:39:56 [scrapy.extensions.logstats] INFO: Crawled 0 pages (at 0 pages/min), scraped 0 items (at 0 items/min)_ 2024-03-21 10:39:56 [scrapy.extensions.telnet] INFO: Telnet console listening on 127.0.0.1:6023 2024-03-21 10:39:56 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.auct.co.th/robots.txt> (referer: None) 2024-03-21 10:39:56 [scrapy.core.engine] DEBUG: Crawled (200) <GET https://www.auct.co.th/products> (referer: None) 2024-03-21 10:39:56 [scrapy.core.engine] INFO: Closing spider (finished) 2024-03-21 10:39:56 [scrapy.statscollectors] INFO: Dumping Scrapy stats: {'downloader/request_bytes': 456, 'downloader/request_count': 2, 'downloader/request_method_count/GET': 2, 'downloader/response_bytes': 25062, 'downloader/response_count': 2, 'downloader/response_status_count/200': 2, 'elapsed_time_seconds': 0.410807, 'finish_reason': 'finished', 'finish_time': datetime.datetime(2024, 3, 21, 3, 39, 56, 863208, tzinfo=datetime.timezone.utc), 'httpcompression/response_bytes': 96141, 'httpcompression/response_count': 2, 'log_count/DEBUG': 5, 'log_count/INFO': 10, 'response_received_count': 2, 'robotstxt/request_count': 1, 'robotstxt/response_count': 1, 'robotstxt/response_status_count/200': 1, 'scheduler/dequeued': 1, 'scheduler/dequeued/memory': 1, 'scheduler/enqueued': 1, 'scheduler/enqueued/memory': 1, 'start_time': datetime.datetime(2024, 3, 21, 3, 39, 56, 452401, tzinfo=datetime.timezone.utc)} 2024-03-21 10:39:56 [scrapy.core.engine] INFO: Spider closed (finished)