Closed 0xking-sol closed 12 months ago
The account cookie and X-Csrf-Token have to match. You need to get both when you manually copy the cookie. This is not the issue though. Twitter requires login now to view/scrape tweets. Even if you want to use account cookies with my scraper, it tries to get the auth cookies before and just sends the "getTweets" requests with account cookies. I think it's currently broken and I'm trying to fix it asap.
0.8 mandates the use of account cookies for viewing tweets. "Fixes" the problem
Thanks for continuously working on this. 0.8 refers to your code version?
I also wrote a simple script for exporting a user's own bookmarks which I'd be happy to add as functionality here if that would be of interest
0.8 indeed does refer to the code version. As Twitter only allows logged-in accounts to view tweets, the use of cookies is mandatory. This comes with a high risk of an account ban. I found a way to scrape tweets without the need for an account but for all other functions, one is still needed. (not included yet) If you want to add your functionality, you can open a pull request, which will be merged after review.
Ran into an error when trying to run the code. Might be a version compatibilty issue, sharing details here.
Approach:
chromedriver_path = "C:/Program Files (x86)/chromedriver.exe" cookies = "" #no account cookies cookies = {'Cookie': 'your Cookie', 'X-Csrf-Token': 'your csrf token'} #with account cookie proxy_list = []
twitter_handle = ["elonmusk"] # single account twitter_handles = ["elonmusk", "POTUS", "latestinspace"] # multiple accounts scraper = TwitterScraper(twitter_handle, chromedriver_path, cookies, proxy_list) tweets = scraper.getTweetsText()
print(tweets)