`GetUserTimeline()` returning `json: cannot unmarshal number into Go value of type anaconda.Contributor`

[joelsimonoff]

Here is how we call GetUserTimeline():

v := url.Values{}

v.Set("count", "1000")
v.Set("screen_name", screenName)

tweets, err := api.GetUserTimeline(v)

We declare api in another function and pass it to the function that gets the users timeline:

api := newAnacondaAPI(u.AuthtokenVal, u.AuthtokenSecretVal)

This is the error we get:

json: cannot unmarshal number into Go value of type anaconda.Contributor

Also we cannot typecast it to type *anaconda.ApiError

This panics:

err, ok := err.(*anaconda.ApiError)
if !ok{
  panic("cannot typecast error")
}

We have done thousands of successful Twitter scans and we just encountered this for the first time tonight.

Any ideas what's causing this? How can I help you guys debug this? This code is running on our production server and we need to fix this ASAP, so no more users are affected.

[ChimeraCoder]

Any ideas what's causing this?

Yes - Twitter's API returns heterogeneous JSON, and they don't always document this, or document changes to the request body structure. I'm sorry you ran into this. :(

This has happened in the past with obscure fields (like scope, used by Twitter to limit visibility of tweets in countries like Iran and China). I'm a little surprised to see it happening to Contributor, but there we have it.

How can I help you guys debug this?

If you're able to give us the specific query that causes the error, that would help. If it's something reproducible on our end (ie, the error isn't dependent on the authenticated user themselves), then we can record the raw response and see if the schema has changed and we need to update it here. (Or, if it's dependent on the authenticated user, if you're able to record the raw response and give it here, that works too).

Lastly, the error won't be of type anaconda.ApiError because it's being given from the encoding/json package itself. It'll be of type json.*UnmarshalTypeError.

[ChimeraCoder]

One more thing - I'm traveling at the moment so may be slow to respond, but will keep an eye on this.

More broadly - part of my plans for anaconda include providing a way to record and re-inspect the response body here, because this is a troublesome (albeit rare) issue. It wasn't built into the original design of anaconda because it didn't start manifesting on Twitter's end until a few years after we originally wrote it. Retroactively inspecting the request is possible in Go, but it'll require making pretty broad API changes in order to handle it seamlessly.

Now that dep is progressing and explicit versioning is becoming more commonplace, I feel more comfortable making these sweeping changes (which will necessitate changes for people not version-pinning to older versions of anaconda).

[joelsimonoff]

Ok, I will record the raw JSON and post it later today. Thanks for your help

[joelsimonoff]

Sorry I have been busy the past few days. I will definitely post it tomorrow.

[ChimeraCoder]

No problem - no rush on my end

[muesli]

@JoelSimonoff Any updates on this one?

[joelsimonoff]

Sorry my company has been growing a bit and I had to switch hats from engineer to business for the last few weeks. Now I am edging back into engineering. I should get to this tomorrow

[muesli]

Thanks, much appreciated!

[joelsimonoff]

Today I was able to get the users credentials and twitter production credentials and get my dev environment ready to simulate our user's experience. We have a pretty strict privacy policy so I had to jump though a few hoops, to do this while complying with privacy policy.

I will start investigating the code tomorrow.

[joelsimonoff]

I forked the repo to start debugging and realized I was a few months behind the newest commit and it looks like this was fixed in a newer commit. Do you want me to investigate more and figure out exactly what the problem was or should we just close the issue?

[ChimeraCoder]

Ah no worries. Let us know if you run into it again!