GetFriendsListAll return only 200 friends

[melihmucuk]

I tried get all friends of user but it returns only 200 of them. How can I get all of them ?

api := getNewAPI(token, tokenSecret)
v := url.Values{}
v.Set("count", "200")
v.Set("skip_status", "true")
friends := <-api.GetFriendsListAll(v)

[Francesco149]

Well, you are setting count to 200, so it's only getting 200 friends as you requested.

See count and cursor parameters here https://dev.twitter.com/rest/reference/get/friends/ids

GetFriendsListAll should already set cursor correctly and re-call the api until the full list has been retrieved, so all you should need to do is:

api := getNewAPI(token, tokenSecret)
v := url.Values{}
v.Set("skip_status", "true")
friends := <-api.GetFriendsListAll(v)

[melihmucuk]

Did you test it ? If you don't set the count, it returns just 20 (default) results.

[Francesco149]

Nah, I'm just speculating. Let's see, the function description says

// Like GetFriendsList, but returns a channel instead of a cursor and pre-fetches the remaining results
// This channel is closed once all values have been fetched

this likely means that you have to keep fetching data from the channel until it is closed, so maybe something like this?

api := getNewAPI(token, tokenSecret)
v := url.Values{}
v.Set("skip_status", "true")
chFriends := api.GetFriendsListAll(v)

var allFriends []anaconda.User
for {
    friends, more := <-chFriends
    allFriends = append(allFriends, friends)
    if !more {
        break
    }
}

[melihmucuk]

Get friends request has request limit on twitter side. How to handle rate limiting ? If I set the count=200 for next 15 request, I can get only 3000 friends. What if the user has more than 3000 friends ? Anyway thanks for your attention. @Francesco149

[Francesco149]

hmm what's the specific use case for this? because most of the time you'll want the user to just browse the friends list page by page and never retrieve all friends at once. anaconda has rate limiting (api.SetDelay) to set a delay between each request.

Additionally, the other api (which only gets friend id's) I linked earlier can retrieve up to 5000 friends in one request https://dev.twitter.com/rest/reference/get/friends/ids and is used by GetFriendsIds

var friendIds []int64
v = url.Values{}
v.Set("count", "5000")
v.Set("screen_name", "user screen name")
next_cursor := "-1"
for next_cursor != "0" {
    v.Set("cursor", next_cursor)
    c, err := a.GetFriendsIds(v)
    if err != nil { log.Fatal(err) }
    friendIds = append(friendIds, c.Ids)
    next_cursor = c.Next_cursor_str
}

(untested code) however if you need more than just the user id you will still need to call GetUsersShowById for each id which has a rate limit of 180 or something like that.

[melihmucuk]

I want to get all friends of the user and insert to db for my apps. I think I should write a background job for this and handle rate limiting on my server side. @Francesco149

[Francesco149]

well you could always just store the id's which is quicker and can get 5000 id's per request. but yeah, if you want to cache ALL information then you're definitely gonna need a background job

[muesli]

Closing as this is a (sensible) limitation of Twitter's API and not a bug in anaconda.