mvabdi / vsco-scraper

Easily allows for scraping a VSCO
MIT License
133 stars 25 forks source link

[Feature Request] Add a option flag to download original profile images #22

Closed lazytownfan closed 2 years ago

lazytownfan commented 2 years ago

I'm wondering if the original profile picture image can be downloaded via an additional option flag (say, via -p or --getProfile and the analogous multiple option -mp or --multipleProfile).

I only recently discovered this CLI tool and it's been a life saver - much easier than going to view-source of every VSCO post and using Ctrl+F to find the full resolution URL after each unique instance of "og:image".

Currently, I use the same process to download profile images, except I use "profileImage" the search query and discard the trailing ?c=1&d=1&w=300 (otherwise, you will be prompted to download a 210 by 210 px square crop).

Previously, I used to right click on the circular profile image in my web browser and select "Open Image in New Tab" and would be prompted to download a really tiny 105 by 105 px crop.

github-userx commented 2 years ago

Message ID: @.***>It’s been a while that I’ve worked with the vsco scraper but just want to mention to you some other tools that might be useful for your scraping (profile pictures etc):

flhdlua commented 2 years ago

It'd be really useful!

mvabdi commented 2 years ago

I will probably add this in, and close this when I do.

flhdlua commented 2 years ago

Hey, since you are willing to improve the app, I have two suggestions that might be interesting:

  1. Scrape only new media since the last scrape. Uses the last modified time of the latest media item in the destination directory to compare. "--latest" or "l"
  2. Get the User ID** and save (maybe beside the username) -The profile crashes after they change the username, with the user ID you can generate a direct link to the profile, regardless of username. "vsco://user/00000000/grid" (in place of zeros you insert the 8-digit user ID) it works on mobile only because it opens directly on the app.

** this ID is in every photo URL

mvabdi commented 2 years ago

Hey @flhdlua, the first feature already exists in the application. The scraper only scrapes images that have not been downloaded by way of the unique id each picture has.

For example, if you delete a picture from the directory it downloads from, it will redownload it.

For the second feature, I just added it in via a "-ch" option flag. This will cache siteids per user.

mvabdi commented 2 years ago

@lazytownfan, I will add that the profile picture capability in a day or two.

flhdlua commented 2 years ago

Thanks so much @mvabdi! It works great, already tested it.

About the first feature, maybe I expressed myself badly. What I meant was to add and option to only download from the last photo onwards... That's because I wish to delete some unusable photos and not have them downloaded again, got it?

Again, thank you so much, you are awesome!!

mvabdi commented 2 years ago

@flhdlua I will look into it. I initially built this out not to store any state. So it just redownloads what is not in the directory.

In any case, I am currently working on the profile picture feature for this feature request.

You can open a feature request if you'd like.

mvabdi commented 2 years ago

@lazytownfan Do you still need this? I had figured out how to get the profile pictures with the username, but I could not figure out how to find them with just the siteid. It would break the cachehit feature since it relies on the siteid. I can still go for it, but let me know if this is still needed.

lazytownfan commented 2 years ago

@mvabdi Yes, but the original feature request only had getting profile pictures with only the username.

I think the part about the siteid is what @flhdlua asked for above.

However, if using the siteid is a more robust approach (i.e., will be able to handle when a user changes their username), then I'd like to see if the siteid approach could work.

mvabdi commented 2 years ago

@lazytownfan Figuring out how to get the profile picture with just the site id was a fun challenge. I should have an update out soon with this capability soon.

mvabdi commented 2 years ago

This is now included! It does not have the same naming sense as most of the other files, this one just uses vsco's unique id for the jpg name.