MartinKBeck / TwitterScraper

Repository containing all files relevant to my basic and advanced tweet scraping articles.
197 stars 117 forks source link

Question about scraping multiple users from a list #10

Closed angelaaaateng closed 1 year ago

angelaaaateng commented 2 years ago

Hi! Thanks for your super helpful Jupyter Notebook and Medium tutorial. Really appreciate the time and effort you put into this! Quick question, how do you scrape multiple users in a list? I would ideally like to iterate through a list of usernames and use your code below:

`# Using TwitterSearchScraper to scrape data and append tweets to list
for i,tweet in enumerate(sntwitter.TwitterSearchScraper('from:jack').get_items()):
    if i>maxTweets:
        break
    tweets_list1.append([tweet.date, tweet.id, tweet.content, tweet.user.username])`

I tried to iterate through my list like below, but I think I'm doing something wrong.

list = [user1, user 2, user3....]
i = 0
for i, tweet in enumerate(sntwitter.TwitterSearchScraper('from:list[i]').get_items()):

Would appreciate any advice! Thank you :)

MartinBeckUT commented 2 years ago

Hey,

With for loops you have to consider how you're applying a function or process. In this one, you're trying to scrape tweets for each user in that list. So it would actually look something like this

list = [user1, user 2, user3....]

for user in list:
     for i, tweet in enumerate(sntwitter.TwitterSearchScraper(f"from:{user}").get_items()): ....

So essentially what's happening is for each user in your list you want to pull all these tweets.

I also personally use f string for modifying strings it's used commonly for its speed and I think makes it easier to format string since you're inputting the variable directly. Anyways, f strings normally look like the below

f"{variable_name}"
So for example you can do something fancy like

user = 'Martin'
formatted_string = f"hello {user}"

print(formatted_string)

Output: hello Martin

Let me know if this makes sense.

Edwardvi commented 2 years ago

thanks! i meet the same question! your answer help me make code run!

tienquyet28 commented 2 years ago

please update the CLI code python in os.system. It seem to be not working with retweetCount, like Count. when i use these CLI code in .inypb it is not working. So please help me. Thank you