hardikvasa / google-images-download

Python Script to download hundreds of images from 'Google Images'. It is a ready-to-run code!
MIT License
8.56k stars 2.1k forks source link

Ubuntu: Error JSONDecodeError when limit > 100 #315

Open vanchung1995 opened 4 years ago

vanchung1995 commented 4 years ago

It works well with limit <= 100 but raise error when limit >= 101, maybe when use chrome driver, it cannot parse main page Detail bellow: Screenshot from 2020-05-22 16-58-53

protrolium commented 4 years ago

I installed chromedriver via homebrew cask tap and then had to specify the full path inside google_images_download.py

browser = webdriver.Chrome("/usr/local/Caskroom/chromedriver/83.0.4103.39/chromedriver", chrome_options=options)

edit: sorry I realized my answer is probably off-topic as it is solution not for ubuntu but for macos

Geremia commented 4 years ago

I get this issue with master (0d2bf8f), regardless of what I set limit to:

$ googleimagesdownload --keywords "car" --limit 1 

Item no.: 1 --> Item name = car
Evaluating...
Starting Download...
Traceback (most recent call last):
  File "/usr/bin/googleimagesdownload", line 11, in <module>
    load_entry_point('google-images-download==2.8.0', 'console_scripts', 'googleimagesdownload')()
  File "/usr/lib64/python3.8/site-packages/google_images_download/google_images_download.py", line 1011, in main
    paths,errors = response.download(arguments)  #wrapping response in a variable just for consistency
  File "/usr/lib64/python3.8/site-packages/google_images_download/google_images_download.py", line 838, in download
    paths, errors = self.download_executor(arguments)
  File "/usr/lib64/python3.8/site-packages/google_images_download/google_images_download.py", line 965, in download_executor
    items,errorCount,abs_path = self._get_all_items(raw_html,main_directory,dir_name,limit,arguments)    #get all image items and download images
  File "/usr/lib64/python3.8/site-packages/google_images_download/google_images_download.py", line 768, in _get_all_items
    image_objects = self._get_image_objects(page)
  File "/usr/lib64/python3.8/site-packages/google_images_download/google_images_download.py", line 758, in _get_image_objects
    image_objects = json.loads(object_decode)[31][0][12][2]
  File "/usr/lib64/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/usr/lib64/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib64/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The pattern matching in _get_image_objects isn't working correctly, because object_raw is an empty string:

    # Getting all links with the help of '_images_get_next_image'
    def _get_image_objects(self,s):
        print(s)
        start_line = s.find("AF_initDataCallback({key: \\'ds:2\\'") - 10
        start_object = s.find('[', start_line + 1)
        end_object = s.find('</script>', start_object + 1) - 4
        object_raw = str(s[start_object:end_object])
        object_decode = bytes(object_raw, "utf-8").decode("unicode_escape")
        image_objects = json.loads(object_decode)[31][0][12][2]
        image_objects = [x for x in image_objects if x[0]==1]
        return image_objects

There is no "AF_initDataCallback" in s.

megrimm commented 4 years ago

Hello,

I am gettin the same error. Was there a fix?

Traceback (most recent call last):
  File "/usr/local/bin/googleimagesdownload", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.8/site-packages/google_images_download/google_images_download.py", line 1006, in main
    paths,errors = response.download(arguments)  #wrapping response in a variable just for consistency
  File "/usr/local/lib/python3.8/site-packages/google_images_download/google_images_download.py", line 833, in download
    paths, errors = self.download_executor(arguments)
  File "/usr/local/lib/python3.8/site-packages/google_images_download/google_images_download.py", line 960, in download_executor
    items,errorCount,abs_path = self._get_all_items(raw_html,main_directory,dir_name,limit,arguments)    #get all image items and download images
  File "/usr/local/lib/python3.8/site-packages/google_images_download/google_images_download.py", line 763, in _get_all_items
    image_objects = self._get_image_objects(page)
  File "/usr/local/lib/python3.8/site-packages/google_images_download/google_images_download.py", line 753, in _get_image_objects
    image_objects = json.loads(object_decode)[31][0][12][2]
  File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/local/Cellar/python@3.8/3.8.5/Frameworks/Python.framework/Versions/3.8/lib/python3.8/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Joeclinton1 commented 4 years ago

Have you tried this?