Open amelnait opened 2 months ago
Hello, I run the code and followed the instructions as mentionned in the readme.md, but I got this result when I run this command : scrapy crawl amazon_search_product :
2024-06-26 14:15:05 [scrapy.statscollectors] INFO: Dumping Scrapy stats: {'downloader/request_bytes': 326, 'downloader/request_count': 1, 'downloader/request_method_count/GET': 1, 'downloader/response_bytes': 457, 'downloader/response_count': 1, 'downloader/response_status_count/403': 1, 'elapsed_time_seconds': 1.994968, 'feedexport/success_count/FileFeedStorage': 1, 'finish_reason': 'finished', 'finish_time': datetime.datetime(2024, 6, 26, 12, 15, 5, 225176, tzinfo=datetime.timezone.utc), 'httpcompression/response_bytes': 291, 'httpcompression/response_count': 1, 'log_count/INFO': 11, 'log_count/WARNING': 1, 'response_received_count': 1, 'scheduler/dequeued': 2, 'scheduler/dequeued/memory': 2, 'scheduler/enqueued': 2, 'scheduler/enqueued/memory': 2, 'start_time': datetime.datetime(2024, 6, 26, 12, 15, 3, 230208, tzinfo=datetime.timezone.utc)} 2024-06-26 14:15:05 [scrapy.core.engine] INFO: Spider closed (finished)
When I tried to store the output in a json file I get an empty array on it. Do you know how to solve this ?
It seems like you are not searching for any products, I was able to reproduce the same issue by passing an empty keyword_list = ['']
However, as soon as I added a search term such as Ipad it worked `
Wadprog
commented
2 months ago 
Hello, I run the code and followed the instructions as mentionned in the readme.md, but I got this result when I run this command : scrapy crawl amazon_search_product :
2024-06-26 14:15:05 [scrapy.statscollectors] INFO: Dumping Scrapy stats: {'downloader/request_bytes': 326, 'downloader/request_count': 1, 'downloader/request_method_count/GET': 1, 'downloader/response_bytes': 457, 'downloader/response_count': 1, 'downloader/response_status_count/403': 1, 'elapsed_time_seconds': 1.994968, 'feedexport/success_count/FileFeedStorage': 1, 'finish_reason': 'finished', 'finish_time': datetime.datetime(2024, 6, 26, 12, 15, 5, 225176, tzinfo=datetime.timezone.utc), 'httpcompression/response_bytes': 291, 'httpcompression/response_count': 1, 'log_count/INFO': 11, 'log_count/WARNING': 1, 'response_received_count': 1, 'scheduler/dequeued': 2, 'scheduler/dequeued/memory': 2, 'scheduler/enqueued': 2, 'scheduler/enqueued/memory': 2, 'start_time': datetime.datetime(2024, 6, 26, 12, 15, 3, 230208, tzinfo=datetime.timezone.utc)} 2024-06-26 14:15:05 [scrapy.core.engine] INFO: Spider closed (finished)
When I tried to store the output in a json file I get an empty array on it. Do you know how to solve this ?