search

Lodour / Weibo-Album-Crawler

A multiprocessing crawler for weibo albums.
MIT License
99 stars 34 forks source link

The album in super topic (超级话题) can not be downloaded #12

Closed CharlesLiu7 closed 5 years ago

CharlesLiu7 commented 5 years ago

I am going to download the whole photos in a super topic, e.g. super topic: 动物, I try to add https://www.weibo.com/p/10080883d7b8a4ba9ca03c1e863198265ea9fa to the TARGETS, and get exception. Because in super topic, the photos are not organized by albums but in waterfall style.

So, how can I download photos in super topic 😭 ?

Lodour commented 5 years ago

You can use the Developer Tools of your browser to figure out:

  1. The actual URL that returns a group of picture's meta-data, like mid, pid.
  2. How such meta-data is converted into the actual URL of the pictures.
CharlesLiu7 commented 5 years ago

I find an api to get the large photo link which is different with your way.

API format:

http://photo.weibo.com/$uid/wbphotos/large/mid/$mid/pid/$pid

the parameters in the API are the photo's uid, mid and pid, and in the API webpage there exists the large photo url.

Thanks a lot!

CharlesLiu7 commented 5 years ago

I write a repo to download the super topic album, it's here

我新建了个repo来下载超级话题的相册，链接。