[Usage] Faster OCR VQA download.

[John-Ge]

Describe the issue

Issue: I find download ocr vqa images are slow and I implement a multi-thread download file.

Command:

import concurrent.futures
def download_image(k):
    ext = os.path.splitext(data[k]['imageURL'])[1]
    outputFile = 'images/%s%s' % (k, ext)

    # Only download the image if it doesn't exist
    if not os.path.exists(outputFile):
        ureq.urlretrieve(data[k]['imageURL'], outputFile)

if download == 1:
    # Create the directory if it doesn't exist
    if not os.path.exists('./images'):
        os.mkdir('./images')

    # Create a thread pool and download the images in parallel
    with concurrent.futures.ThreadPoolExecutor() as executor:
        executor.map(download_image, data.keys())

This maybe useful.

[xulinui]

https://blog.csdn.net/weixin_46460463/article/details/135537718?spm=1001.2014.3001.5501

[master-chou]

thanks ！！！！

[cooleel]

https://blog.csdn.net/weixin_46460463/article/details/135537718?spm=1001.2014.3001.5501

Thanks for posting the method, but some of the images that I downloaded are damaged. Did anyone face the same issue?