Closed lazytownfan closed 2 years ago
Message ID: @.***>It’s been a while that I’ve worked with the vsco scraper but just want to mention to you some other tools that might be useful for your scraping (profile pictures etc):
gallery-dl (python Commandline)
RipMe (Java GUI)
jDownloader (Java GUI)
It'd be really useful!
I will probably add this in, and close this when I do.
Hey, since you are willing to improve the app, I have two suggestions that might be interesting:
** this ID is in every photo URL
Hey @flhdlua, the first feature already exists in the application. The scraper only scrapes images that have not been downloaded by way of the unique id each picture has.
For example, if you delete a picture from the directory it downloads from, it will redownload it.
For the second feature, I just added it in via a "-ch" option flag. This will cache siteids per user.
@lazytownfan, I will add that the profile picture capability in a day or two.
Thanks so much @mvabdi! It works great, already tested it.
About the first feature, maybe I expressed myself badly. What I meant was to add and option to only download from the last photo onwards... That's because I wish to delete some unusable photos and not have them downloaded again, got it?
Again, thank you so much, you are awesome!!
@flhdlua I will look into it. I initially built this out not to store any state. So it just redownloads what is not in the directory.
In any case, I am currently working on the profile picture feature for this feature request.
You can open a feature request if you'd like.
@lazytownfan Do you still need this? I had figured out how to get the profile pictures with the username, but I could not figure out how to find them with just the siteid. It would break the cachehit feature since it relies on the siteid. I can still go for it, but let me know if this is still needed.
@mvabdi Yes, but the original feature request only had getting profile pictures with only the username.
I think the part about the siteid is what @flhdlua asked for above.
However, if using the siteid is a more robust approach (i.e., will be able to handle when a user changes their username), then I'd like to see if the siteid approach could work.
@lazytownfan Figuring out how to get the profile picture with just the site id was a fun challenge. I should have an update out soon with this capability soon.
This is now included! It does not have the same naming sense as most of the other files, this one just uses vsco's unique id for the jpg name.
I'm wondering if the original profile picture image can be downloaded via an additional option flag (say, via
-p
or--getProfile
and the analogous multiple option-mp
or--multipleProfile
).I only recently discovered this CLI tool and it's been a life saver - much easier than going to
view-source
of every VSCO post and using Ctrl+F to find the full resolution URL after each unique instance of"og:image"
.Currently, I use the same process to download profile images, except I use
"profileImage"
the search query and discard the trailing?c=1&d=1&w=300
(otherwise, you will be prompted to download a 210 by 210 px square crop).Previously, I used to right click on the circular profile image in my web browser and select "Open Image in New Tab" and would be prompted to download a really tiny 105 by 105 px crop.