Followers Endpoint Returning Inconsistent Results And Other Bugs...

[macharborguy]

I reposted this to the old issue, but with that issue being closed back in 2015 (https://github.com/justintv/Twitch-API/issues/320), i do not know if it will be revisited in a timely matter, especially with V5 in the works right now, so I will repost my information here...

---

I do not know if this is indented or not, but occationally the cursor value is not appearing in the "next" url. In addition, I am receiving Status Code 400 Bad Request messages against saying that the "offset" value cannot go above 1600.

I am using the "_links.next" parameter as suggested above. Sometimes the cursor value appears in that url, sometimes it does not.

for the status code 400 warning about the incorrect offset value, the cursor value WAS part of the url.

i only noticed this issue today when I restarted our channel bot. We are properly including our client ID in the header.

We only do full checks of our full follower list every 6 hours when our channel is off air, and we do this to make sure we have an up to date list of not only who is following us, but if anyone has unfollowed.

Please look into this issue.

Again, even with the cursor value included in the URL AND using the _links.next parameter from the request results, the warning about the offset value being too high is appearing again.

---

UPDATE: here is the status URL in question:

{"error":"Bad Request","status":400,"message":"The parameter \"offset\" was malformed: the value must be less than or equal to 1600"}

and here was the URL from the previously received _links.next entry:

https://api.twitch.tv/kraken/channels/<channel_name>/follows?channel=<channel_name>&direction=desc&limit=100&offset=1700

Notice the lack of a "cursor" entry.

I edited my code to manually add the cursor value to the end of the URL, potentially doubling up the cursor entry in some URLs. However now sometimes, almost seemingly at random, the _links.next URL changes the "offset" value back to 100, instead of moving to the next set. I say "seemingly at random" because in testing it has happened at the 300 record mark, 1200 mark, and 1400 mark.

---

So at this point I am basically stuck and at a loss. I cannot fix the current issues, because the API is reporting back such inconsistent information and almost seemingly has reverted back to its 2014-2015 "pre-cursor, 1600 follower" limits for no real nor announced reason.

BTW, I have also tested this with V5, and the 1600 limit exists there as well, even when using the cursor value.

https://api.twitch.tv/kraken/channels/37614826/follows?channel_id=8608782&cursor=1437933971523699000&api_version=5&offset=2000&limit=100

returns...

{"error":"Bad Request","status":400,"message":"The parameter \"offset\" was malformed: the value must be less than or equal to 1600"}

Again, this issue was apparently fixed back in 2015 with the addition of the "cursor" value, since many users need the whole list of followers to their channel for things like Minecraft server access and other features that depend on Follower status. (https://github.com/justintv/Twitch-API/issues/320)

[brentlapierre]

Once you've reached the 1600 offset limit, you're supposed to grab the cursor value and go back to an offset of 0 with the new cursor value, thus getting the next set of followers. It's working as intended.

[macharborguy]

So I went back and reviewed my code, as well as made a very basic "follower" gatherer script in NodeJS, and have come to the conclusion that you are incorrect, it is not working as intended.

My reasoning...

Offset, by what is stated in the V3 documentation, is deprecated, and should not be used.

(deprecated) Object offset for pagination. Default is 0. Maximum is 1600.

in addition, the description of the Cursor value...

Twitch uses cursoring to paginate long lists of followers. Check _cursor in response body and set cursor to this value to get the next page of results, or use _links.next to navigate to the next page of results.

So, following the V3 documentation, i wrote up two tests. One using _cursor and the other using _links.next.

When using _links.next, the cursor value is NOT appended to the URL, and instead the _links.next value simply appended the (deprecated) offset value and ITSELF sets the offset value to 1700, violating the spec for this endpoint.

That being said, _links are being deprecated as well in favor for _cursor.

Doing a test JUST using the cursor value to get the next page, I WAS able to get the full list.

---

The fact that my bot lost the ability, out of nowhere, about 2 weeks ago, ADD to the fact that _links.next is being deprecated, it almost seems like Twitch did something while working on the V5 implementation that caused the _links.next method of pagination to revert to the previous (1600) limit.

Again, my bot has been using the same follower gathering code for about a year now, using one of the two methods suggested by Twitch (_links.next), and that JUST now failed a year later, roughly around the same time as the announcements for V5.

---

in closing, for anyone looking to gather every single follower of theirs using a script, make sure you grab the _cursor value from each page, add that to the URL, as that will then request the next page, and so on down the line. Do NOT use _links.next or offset, because as of V5 those will not be reliable. _links.next is ALREADY NOT reliable in V3.