Closed xinxinxing closed 1 year ago
Thanks for your feedback. You mean only 25% pictures can be downlowded or only 25% downloaded picture can be opened? If your problem is the former, you can try to decrease the number of the threads, and the download process is also influenced by your network. And another possible reason is that the website forbade your ip as it detected a lot of requests from the same ip, so you may need a ip proxy pool for your web crawler.
We meet the same problem, there are a lot of URLs that can not be downloaded as follows. The page is redirected to "https://www.baidu.com/search/error.html".
https://img0.baidu.com/it/u=1182417238,35690091&fm=253&fmt=auto&app=138&f=JPG?w=375&h=500 https://img2.baidu.com/it/u=785589976,1330059829&fm=15&fmt=auto https://img1.baidu.com/it/u=4031456945,47925087&fm=15&fmt=auto https://img1.baidu.com/it/u=3799845611,2090883687&fm=26&fmt=auto https://img1.baidu.com/it/u=1876254937,3975911663&fm=26&fmt=auto https://img0.baidu.com/it/u=1738521458,3601161552&fm=26&fmt=auto https://img0.baidu.com/it/u=1828754748,3422674016&fm=26&fmt=auto https://img1.baidu.com/it/u=2158967391,1486838041&fm=26&fmt=auto http://t14.baidu.com/it/u=134094118,4184559187&fm=224&app=112&f=JPEG?w=500&h=500 https://img0.baidu.com/it/u=3728162256,1714271687&fm=253&fmt=auto&app=120&f=JPEG?w=500&h=659 https://img0.baidu.com/it/u=228045818,2153011546&fm=224&fmt=auto&gp=0.jpg https://img1.baidu.com/it/u=1980676238,2500877389&fm=26&fmt=auto https://img1.baidu.com/it/u=1599521237,3536696877&fm=253&fmt=auto&app=120&f=JPEG?w=700&h=494 https://img1.baidu.com/it/u=266610651,3644722975&fm=26&fmt=auto https://img2.baidu.com/it/u=2090927373,2805419862&fm=15&fmt=auto https://img1.baidu.com/it/u=1556693551,3383313790&fm=15&fmt=auto https://img1.baidu.com/it/u=1626134414,3266492804&fm=253&fmt=auto&app=138&f=JPEG?w=667&h=500 https://img2.baidu.com/it/u=1923152243,2877920931&fm=26&fmt=auto https://img2.baidu.com/it/u=3043109430,776096273&fm=26&fmt=auto https://img2.baidu.com/it/u=2791924693,3461582583&fm=26&fmt=auto https://img0.baidu.com/it/u=1927151116,2201870308&fm=26&fmt=auto https://img0.baidu.com/it/u=2424599050,3783123750&fm=224&fmt=auto&gp=0.jpg https://img1.baidu.com/it/u=188843659,3715987422&fm=224&fmt=auto&gp=0.jpg https://img2.baidu.com/it/u=507184859,301561173&fm=15&fmt=auto https://img0.baidu.com/it/u=4255270059,849811271&fm=26&fmt=auto https://img0.baidu.com/it/u=1861832780,438212316&fm=253&fmt=auto&app=138&f=JPEG?w=667&h=500 https://img1.baidu.com/it/u=2468189706,1866533212&fm=15&fmt=auto https://img2.baidu.com/it/u=1205031373,4055638944&fm=15&fmt=auto https://img2.baidu.com/it/u=3995564558,1063022454&fm=15&fmt=auto https://img1.baidu.com/it/u=888992645,111571178&fm=26&fmt=auto https://img0.baidu.com/it/u=3477598855,3485579616&fm=26&fmt=auto&gp=0.jpg //img10.360buyimg.com/n7/jfs/t1/189274/18/11029/135960/60dc56f4Ec7bd26a1/d23205932f2badae.jpg https://t8.baidu.com/it/u=1907321350,2717707273&fm=167&fmt=auto&app=43&f=JPEG?w=650&h=414&s=7F26DD4FE472099C2751190A03003012 https://img0.baidu.com/it/u=2246122100,2496406693&fm=253&fmt=auto&app=138&f=JPEG?w=500&h=375 https://img2.baidu.com/it/u=2118740241,2169880236&fm=15&fmt=auto https://img0.baidu.com/it/u=1973621471,2214051361&fm=15&fmt=auto https://img0.baidu.com/it/u=2332285066,2587573021&fm=26&fmt=auto https://img2.baidu.com/it/u=785792451,2497929388&fm=253&fmt=auto&app=120&f=JPEG?w=523&h=500 https://img1.baidu.com/it/u=436828074,3229837550&fm=26&fmt=auto //img10.360buyimg.com/n7/jfs/t1/180463/31/12566/139912/60e16734E83ee28ad/fcf079fcbf07d513.jpg https://img2.baidu.com/it/u=543282838,304192473&fm=15&fmt=auto https://img1.baidu.com/it/u=1723029811,18867895&fm=26&fmt=auto https://img1.baidu.com/it/u=2440390041,1416840750&fm=15&fmt=auto https://img2.baidu.com/it/u=2276748736,172815922&fm=26&fmt=auto http://t13.baidu.com/it/u=1098180671,3255145601&fm=224&app=112&f=JPEG?w=350&h=350&s=21900C7163CF435F40A922780300C066 https://img2.baidu.com/it/u=3829992130,722960501&fm=26&fmt=auto https://img0.baidu.com/it/u=3318518452,790623393&fm=15&fmt=auto https://img1.baidu.com/it/u=1708143803,1834618559&fm=224&fmt=auto&gp=0.jpg https://img2.baidu.com/it/u=3367593861,4277447787&fm=26&fmt=auto https://img2.baidu.com/it/u=606678418,464588331&fm=253&fmt=auto&app=120&f=PNG?w=606&h=402
https://img0.baidu.com/it/u=1182417238,35690091&fm=253&fmt=auto&app=138&f=JPG?w=375&h=500 https://img2.baidu.com/it/u=785589976,1330059829&fm=15&fmt=auto https://img1.baidu.com/it/u=4031456945,47925087&fm=15&fmt=auto https://img1.baidu.com/it/u=3799845611,2090883687&fm=26&fmt=auto https://img1.baidu.com/it/u=1876254937,3975911663&fm=26&fmt=auto https://img0.baidu.com/it/u=1738521458,3601161552&fm=26&fmt=auto https://img0.baidu.com/it/u=1828754748,3422674016&fm=26&fmt=auto https://img1.baidu.com/it/u=2158967391,1486838041&fm=26&fmt=auto http://t14.baidu.com/it/u=134094118,4184559187&fm=224&app=112&f=JPEG?w=500&h=500 https://img0.baidu.com/it/u=3728162256,1714271687&fm=253&fmt=auto&app=120&f=JPEG?w=500&h=659 https://img0.baidu.com/it/u=228045818,2153011546&fm=224&fmt=auto&gp=0.jpg https://img1.baidu.com/it/u=1980676238,2500877389&fm=26&fmt=auto https://img1.baidu.com/it/u=1599521237,3536696877&fm=253&fmt=auto&app=120&f=JPEG?w=700&h=494 https://img1.baidu.com/it/u=266610651,3644722975&fm=26&fmt=auto https://img2.baidu.com/it/u=2090927373,2805419862&fm=15&fmt=auto https://img1.baidu.com/it/u=1556693551,3383313790&fm=15&fmt=auto https://img1.baidu.com/it/u=1626134414,3266492804&fm=253&fmt=auto&app=138&f=JPEG?w=667&h=500 https://img2.baidu.com/it/u=1923152243,2877920931&fm=26&fmt=auto https://img2.baidu.com/it/u=3043109430,776096273&fm=26&fmt=auto https://img2.baidu.com/it/u=2791924693,3461582583&fm=26&fmt=auto https://img0.baidu.com/it/u=1927151116,2201870308&fm=26&fmt=auto https://img0.baidu.com/it/u=2424599050,3783123750&fm=224&fmt=auto&gp=0.jpg https://img1.baidu.com/it/u=188843659,3715987422&fm=224&fmt=auto&gp=0.jpg https://img2.baidu.com/it/u=507184859,301561173&fm=15&fmt=auto https://img0.baidu.com/it/u=4255270059,849811271&fm=26&fmt=auto https://img0.baidu.com/it/u=1861832780,438212316&fm=253&fmt=auto&app=138&f=JPEG?w=667&h=500 https://img1.baidu.com/it/u=2468189706,1866533212&fm=15&fmt=auto https://img2.baidu.com/it/u=1205031373,4055638944&fm=15&fmt=auto https://img2.baidu.com/it/u=3995564558,1063022454&fm=15&fmt=auto https://img1.baidu.com/it/u=888992645,111571178&fm=26&fmt=auto https://img0.baidu.com/it/u=3477598855,3485579616&fm=26&fmt=auto&gp=0.jpg //img10.360buyimg.com/n7/jfs/t1/189274/18/11029/135960/60dc56f4Ec7bd26a1/d23205932f2badae.jpg https://t8.baidu.com/it/u=1907321350,2717707273&fm=167&fmt=auto&app=43&f=JPEG?w=650&h=414&s=7F26DD4FE472099C2751190A03003012 https://img0.baidu.com/it/u=2246122100,2496406693&fm=253&fmt=auto&app=138&f=JPEG?w=500&h=375 https://img2.baidu.com/it/u=2118740241,2169880236&fm=15&fmt=auto https://img0.baidu.com/it/u=1973621471,2214051361&fm=15&fmt=auto https://img0.baidu.com/it/u=2332285066,2587573021&fm=26&fmt=auto https://img2.baidu.com/it/u=785792451,2497929388&fm=253&fmt=auto&app=120&f=JPEG?w=523&h=500 https://img1.baidu.com/it/u=436828074,3229837550&fm=26&fmt=auto //img10.360buyimg.com/n7/jfs/t1/180463/31/12566/139912/60e16734E83ee28ad/fcf079fcbf07d513.jpg https://img2.baidu.com/it/u=543282838,304192473&fm=15&fmt=auto https://img1.baidu.com/it/u=1723029811,18867895&fm=26&fmt=auto https://img1.baidu.com/it/u=2440390041,1416840750&fm=15&fmt=auto https://img2.baidu.com/it/u=2276748736,172815922&fm=26&fmt=auto http://t13.baidu.com/it/u=1098180671,3255145601&fm=224&app=112&f=JPEG?w=350&h=350&s=21900C7163CF435F40A922780300C066 https://img2.baidu.com/it/u=3829992130,722960501&fm=26&fmt=auto https://img0.baidu.com/it/u=3318518452,790623393&fm=15&fmt=auto https://img1.baidu.com/it/u=1708143803,1834618559&fm=224&fmt=auto&gp=0.jpg https://img2.baidu.com/it/u=3367593861,4277447787&fm=26&fmt=auto https://img2.baidu.com/it/u=606678418,464588331&fm=253&fmt=auto&app=120&f=PNG?w=606&h=402
Thanks for your feedback. We will check it and give feedback later.
It seems that some urls are expired. We will try other methods to make the data public. Thanks for your attention.
It seems even worse now, only few url is available after i try several urls, much lower than 25% of all provided i think. Hoping for your new release and thanks for your work.
I have the same problem
any news for the expired urls?
any news for the expired urls?
Yes, after this Chinese Spring Festival, we will directly provide the images.
any news for new urls?
Yes!!!! This week, We will provide a link to directly down load all the images . thanks for your attention.
TaiSu data can be freely download now! If you come across any problem, please feel free to give us a feedback. Thanks in advance.
thanks for your effort~~~~
Hello, I would like to ask why many pictures cannot be opened using the official download script. The success rate is only 25 %