hellock / icrawler

A multi-thread crawler framework with many builtin image crawlers provided.
http://icrawler.readthedocs.io/en/latest/
MIT License
854 stars 174 forks source link

your_img_dir of icrawler doesn't work in my code. #69

Closed LimJoongSub closed 4 years ago

LimJoongSub commented 4 years ago
from icrawler.builtin import GoogleImageCrawler

google_crawler = GoogleImageCrawler(
    feeder_threads=1,
    parser_threads=1,
    downloader_threads=4,
    storage={'root_dir': 'C:\\Users\Park\PycharmProjects\\test_img'})
filters = dict(
    date=((2017, 1, 1), (2020, 3, 1)))

google_crawler.crawl(keyword='cat', filters=filters, offset=0, max_num=30,
                     max_size=None, file_idx_offset=0)

that is my code. i was looking for how to change the 'your_img_dir' arg. and someone said that just put my dir for saving the downloaded image from google. and then dont change 'root_dir'. So i just follow that.

My code show me 2020-03-17 16:04:46,173 - INFO - icrawler.crawler - Crawling task done! but there is no downloaded image at 'C:\Users\Park\PycharmProjects\test_img'

there is any problem? or Am i doing wrong way?

wutchy commented 4 years ago

I encountered the same problem.

65 maybe answer the problem.

Luke256 commented 4 years ago

I encountered the same problem.

ajax-lives commented 4 years ago

I used to use iCrawler on a daily basis and I decided to to make a tkinter gui for it yesterday, I think iCrawler no longer works. No idea why, could be something to do with Google? Very disappointing, was a very good, well made package.

LeonhardEuler commented 4 years ago

I used to use iCrawler on a daily basis and I decided to to make a tkinter gui for it yesterday, I think iCrawler no longer works. No idea why, could be something to do with Google? Very disappointing, was a very good, well made package.

I'm facing the same issue, I think Google changed search results page source and the parser no longer works. The downloader queue is always empty.

Bing still works though.

vishal2612200 commented 4 years ago

I used to use iCrawler on a daily basis and I decided to to make a tkinter gui for it yesterday, I think iCrawler no longer works. No idea why, could be something to do with Google? Very disappointing, was a very good, well made package.

I'm facing the same issue, I think Google changed search results page source and the parser no longer works. The downloader queue is always empty.

Bing still works though.

Ya Same thing happening with my code now. BingCrawler is working fine but GoogleCrawler is not working now, It doesn't return any file on any parameter.

whria78 commented 4 years ago

GoogleCrawler still does not work. Any idea ? or other library to be recommended?

ZhiyuanChen commented 4 years ago

Resolved in #84