vladkens / twscrape

2024! X / Twitter API scrapper with authorization support. Allows you to scrape search results, User's profiles (followers/following), Tweets (favoriters/retweeters) and more.
https://pypi.org/project/twscrape/
MIT License
1.12k stars 133 forks source link

2024-05-18 18:44:35.238 | INFO | twscrape.accounts_pool:get_for_queue_or_wait:301 - No account available for queue "Following". Next available at 18:58:18 #192

Closed array-drian closed 6 months ago

array-drian commented 6 months ago

I keep getting this error. I used to do everything in a For Loop before but that wasnt practical so I started using async Tasks. As you can see my account hasnt made a single request. I tried deleting and adding it multiple time. I also logged into my account and it is not suspended. Screenshot_3

#-------------------Process-------------------

async def process_user(api, user_id):
    followings = await gather(api.following(user_id, limit=10000))

    friends = []
    for friend in followings:  # NewFollowingList
        friends.append((str(friend.id), friend.username, friend.url))

    currentFollowing = getCurrentFollowingsForUser(user_id)  # CurrentFollowingList
    allFollowedUsers = getCurrentFollowedUsers()  # AllFollowedUsers

    # Add new identified followings to 'followed_users'
    if updateNeeded(set(allFollowedUsers), set(friends)):
        updateFollowedUsers(set(allFollowedUsers), set(friends))

    # Update 'followed_trackings'
    if updateNeeded(set(currentFollowing), set(friends)):
        updateTrackedFollowings(set(currentFollowing), set(friends), user_id)

#-------------------Main-------------------

async def main():
    api = API()

    while True:
        api.proxy = 'http://' + getRandomProxie()
        tracked_users = getActivelyTrackedUsers()
        user_ids = [user[0] for user in tracked_users]
        tasks = [process_user(api, user_id) for user_id in user_ids]
        await asyncio.gather(*tasks)
        print('I am still up and running...')
        time.sleep(60)

if __name__ == "__main__":
    asyncio.run(main())
array-drian commented 6 months ago

And Yes I know my limit is high. I want to track all followings and it has worked flawlessly before I started using tasks and changing it now doesnt make a difference.

array-drian commented 6 months ago

If I do it like that it works fine

#-------------------Main-------------------

async def main():
    api = API()

    #while True:
    #    api.proxy = 'http://' + getRandomProxie()
    #    tracked_users = getActivelyTrackedUsers()
    #    user_ids = [user[0] for user in tracked_users]
    #    tasks = [process_user(api, user_id) for user_id in user_ids]
    #    await asyncio.gather(*tasks)
    #    print('I am still up and running...')
    #    time.sleep(60)

    user_id = 1688372431

    followings = await gather(api.following(user_id, limit=10000))

    friends = []
    for friend in followings:  # NewFollowingList
        friends.append((str(friend.id), friend.username, friend.url))

    currentFollowing = getCurrentFollowingsForUser(user_id)  # CurrentFollowingList
    allFollowedUsers = getCurrentFollowedUsers()  # AllFollowedUsers

    # Add new identified followings to 'followed_users'
    if updateNeeded(set(allFollowedUsers), set(friends)):
        updateFollowedUsers(set(allFollowedUsers), set(friends))

    # Update 'followed_trackings'
    if updateNeeded(set(currentFollowing), set(friends)):
        updateTrackedFollowings(set(currentFollowing), set(friends), user_id)

if __name__ == "__main__":
    asyncio.run(main())