mvabdi / vsco-scraper

Easily allows for scraping a VSCO
MIT License
132 stars 25 forks source link

Script No Longer Works #32

Closed intothevoid33 closed 2 years ago

intothevoid33 commented 2 years ago

When running the script, this error now appears:

Traceback (most recent call last): File "C:\Python310\Scripts\vsco-scraper-script.py", line 33, in sys.exit(load_entry_point('vsco-scraper==0.65', 'console_scripts', 'vsco-scraper')()) File "C:\Python310\lib\site-packages\vscoscrape\vscoscrape.py", line 763, in main scraper = Scraper(args.username) File "C:\Python310\lib\site-packages\vscoscrape\vscoscrape.py", line 25, in init self.uid = self.session.cookies.get_dict()["vs"] KeyError: 'vs'

bialyrycerz commented 2 years ago

I am getting this same error.

ilovescraperz commented 2 years ago

Same error as OP. Hope this can be worked around

faiblesse2 commented 2 years ago

Okay not just me, thought i had gotten IP blocked

koriekhov commented 2 years ago

Yep, have same error

christoarmani commented 2 years ago

I'm getting the same error

SoulSeek2 commented 2 years ago

yeah same here.. strange thing is that it already happened yesterday for a while. i usually let it run through a list of accounts and yesterday it already said "a crashed b crashed .. and so on" but a while later it worked again. today the same thing and if i scrape a single account i get the above mentioned error message

lazytownfan commented 2 years ago

Side note: could this possibly be related to Issue #31?

mvabdi commented 2 years ago

The endpoints in use on this script, and most other non-selenium based scripts, are over 5 and a half years old. It is kind of a miracle that the original version of this script worked with minor revisions until now.

The last time I added features here, I noticed that vsco did not appear to be using the endpoints any more. They have architected their website differently in the years since.

This issue is happening because an important cookie "['vs']" is no longer being returned by that endpoint. That cookie is used in the endpoint from issue #31, which also appears to not exist anymore.

I think it's unlikely that it is temporary that these apis are down, given that the website still works. For these scrapers to work again essentially involves rewriting their scripts to work with the new logic.

intothevoid33 commented 2 years ago

Thanks for the update. Will this script be updated, or has it run its course?

SilverMight commented 2 years ago

I left a pull request (#33) that seemed to work with VSCO's API changes.

intothevoid33 commented 2 years ago

Thank you, SilverMight! I can confirm that those changes bring the script back to live. Kudos!

ttp976633 commented 2 years ago

as a complete beginner who is just learning the basics, could you explain how I can update the code? I previously used the VSCO Downloader executable file, but I was able to install the vsco-scraper package using the cmd line. Can you explain it to me like I'm 5 how I need to update to fix the script so it works properly? Thanks in advance

christoarmani commented 2 years ago

as a complete beginner who is just learning the basics, could you explain how I can update the code? I previously used the VSCO Downloader executable file, but I was able to install the vsco-scraper package using the cmd line. Can you explain it to me like I'm 5 how I need to update to fix the script so it works properly? Thanks in advance

I would run the following command pip install vsco-scraper --upgrade in the command line to update vsco-scraper. This can also be found under Getting Started.

lazytownfan commented 2 years ago

With this method, updating via pip can only be done after PR #33 is merged.

SilverMight commented 2 years ago

as a complete beginner who is just learning the basics, could you explain how I can update the code? I previously used the VSCO Downloader executable file, but I was able to install the vsco-scraper package using the cmd line. Can you explain it to me like I'm 5 how I need to update to fix the script so it works properly? Thanks in advance

I'd remove your existing install for now (until it's updated) by doing pip uninstall vsco-scraper. Clone or just download as a zip and extract this repository with the fixes. Enter the directory and run python3 setup.py install and it should install the new version. You may want to use a virtual environment, but use your discretion for that one.

mvabdi commented 2 years ago

🤦‍♂️, seems like I over thought it. It is cool to me all this works with minimal changes years later.

Thanks @SilverMight for the PR, and @HuggableSquare for having done the heavy work months prior.

The fixes are merged at this time on version 0.7