search

symphonycms / remote_datasource

An improved datasource to fetch data from TXT, CSV, XML and JSON sources.
Other
14 stars 9 forks source link

Unable to retrieve content from Tender API #5

Closed tonyarnold closed 11 years ago

tonyarnold commented 12 years ago

I'm getting an odd response from the Tender JSON API — it's returning a HTTP response code of 0 when called by this extension. Scooping the HTTP data, it looks like the connection is reset by a peer.

Calling the end point manually using curl works fine, so the problem does appear to lie in our code, not their service.

Any ideas?

nilshoerrmann commented 12 years ago

I remember that we had this problem on an install and a friend found out that the reason was a timeout. It just took too long to fetch the data and so there wasn't even a response code yet. For some reasons, Remote Data Source and Dynamic Data Source didn't catch the timeout but threw an error with the status code 0. It had either to do with how that status variable was set or with the way errors were handled – I don't know anymore.

tonyarnold commented 12 years ago

If this is the case, then it's something that should probably be fixed in the Gateway class in Symphony itself. I'll do some digging as I get the time. Thanks, @nilshoerrmann!

tonyarnold commented 12 years ago

I've found the issue. It seems that the following is being set in my newly created datasources:

public $dsParamTIMEOUT = 1;

It should be:

public $dsParamTIMEOUT = 6;

Or not set at all.

nilshoerrmann commented 12 years ago

But this does only make it less likely that the Gateway return status code 0, right. Remote Data Source should also handle the logged error better: if I'm not mistaken, status code 0 is only returned when the connection times out too early. So the extension should log a timeout and not a status code error (at least I remember that the given error message was the reason in our project why no one understood why this wasn't working).

tonyarnold commented 12 years ago

Absolutely, @nilshoerrmann — I agree. The error messages here were pretty useless — curl doesn't actually return much of value here (although the 0 HTTP status is something we could intercept ourselves, I guess).

brendo commented 12 years ago

Hm, a status code of 0 apparently means 'All good', whereas a status code of 28 means timeout according to this reference.

The Remote DS takes this into account, perhaps it shouldn't..

nilshoerrmann commented 12 years ago

Maybe the problem is that the connection is okay (0 as you said) but that no data has come in yet? Still, Remote Data Source thinks there should be data (maybe it's just checking for the status message) and then fails because actually there isn't any data?