search

gazugafan / chirp

Automatically follow Twitter users and later unfollow them if they don't follow you back. Easily grow your Twitter audience!
MIT License
21 stars 1 forks source link

1 Question, 1 item worth noting #3

Closed IADDIC closed 5 years ago

IADDIC commented 5 years ago

Hi Ken,

Question: Once the process to import an account's followers has started how do I kill the process.

Observation: I a large account is selected to capture the followers, a rate limit is immediately applied and it doesn't look like any id's are captured so that when the process restarts it picks up where it left off.

My use case was to be this: We run a charity called Shelter The World that relies on sponsors from real estate professionals. Who better to harness than the followers of the national real estate association...

Apparently, there are too many id's to capture. The to_follow file never gets written to (or created if this is the first attempt at importing)

twitter account I tried: nardotrealtor size: 301K

Respecting rate limit on the followers_ids method. This will clear at Saturday, Mar 30, 2019 @ 03:19pm UTC proje485@m48 [~/public_html/sheltertheworldorg/real-estate-bot]# Respecting rate limit on the followers_ids method. This will clear at Saturday, Mar 30, 2019 @ 03:19pm UTC

Thanks, Rich

gazugafan commented 5 years ago

Hmm... that could be a tough one to fix at the moment. Twitter breaks up a query like that into pages of, I think 200 results at a time. So, 301,000 users divided by 200 = 1505 API requests. If you hit a rate limit during one of those requests (and with that many, you almost certainly will), the process is simply aborted, and no users ever get added. You can hit CTRL+C if you're sitting there waiting for the process to finish, but if you see that error message then it's already finished and there's nothing else to kill.

A better way to have engineered this would be to add each page of results as we get them, instead of waiting for all the pages. And even better, save where we're at if we hit a rate limit and automatically pick back up where we left off when it clears.

I might revamp this eventually, but it's not a small task. In the meantime I guess you'll have to only add people with less followers. Alternatively, if you have a list of user IDs you can always add them to the to_follow file manually. Note that this needs to be user IDs, though--not screen names.

IADDIC commented 5 years ago

Ohh, I didn't expect you to fix anything but I wanted to share what I tried to do and to be certain I was understanding what you intended. I've already begun to locate smaller lists from local realtors who only have a few thousand followers/agent...at some point I'll scrape the user ID's from the account I was looking at and then enter them into the to_follow file.

I am sure most folks wouldn't think of following an account with 300k+ followers but for me, it was laser-focused on the type of people I want!

As for the schedule...I'll use cron. The way you wrote the 5 minute time I wasn't sure how to change it. I scanned the laravel page...thanks, I'll study it soon.

I appreciate you replying back, especially on the weekend. Be well! If you want feedback when you are ready to update, don't hesitate to reach out.

Regards, Rich