search

twintproject / twint

An advanced Twitter scraping & OSINT tool written in Python that doesn't use Twitter's API, allowing you to scrape a user's followers, following, Tweets and more while evading most API limitations.
MIT License
15.75k stars 2.72k forks source link

Deprecated selectors? #656

Closed alvarotrigo closed 4 years ago

alvarotrigo commented 4 years ago

Description of Issue

When looking at the code of this project, at the tweet.py file, I can see it is using some selector:

    t.user_id_str = tw["data-user-id"]
    t.username = tw["data-screen-name"]

Or in users.py it uses:

        group = ur.find("div", "user-actions btn-group not-following ")

But those selectors don't exist anymore on the mobile view of twitter site. How is it even working?

pielco11 commented 4 years ago

Twint does not scrape Twitter

It replicates the requests that your browser would do like: https://twitter.com/i/search/timeline?vertical=default&src=unkn&include_available_features=1&include_entities=1&max_position=-1&reset_error_state=false&f=tweets&q=%20from%3Ausername

if you run twint with --debug, it creates two log files. One about the requests made, and the other for the content of the latest request

alvarotrigo commented 4 years ago

Thanks.