MartinsOnuoha / countriesNowAPI

CountriesNow is an Open source API for retrieving geo-information for countries, including their states, cities, population, etc. 🌎
http://countriesnow.space
MIT License
185 stars 59 forks source link

Multiple Redirects #105

Closed jasteyn07 closed 2 years ago

jasteyn07 commented 2 years ago

Good day,

It would seem that the API for the Posts to get City with Region and Region in country has changed with more redirects than before. This looks to have changed in the past week. I use Bubble.io and the Bubble.io API, which allows for less than 10 redirects but it would seem you have more than 10. Can you confirm if changes were made and if this could be corrected as it was previously?

Curl: curl --request POST 'https://countriesnow.space/api/v0.1/countries/state/cities' \ --header 'Content-Type: application/json' \ --data-raw '{ "country": "south africa", "state": "gauteng" }'

Error:

**There was an issue setting up your call.

Raw error for the API Exceeded maxRedirects. Probably stuck in a redirect loop https://countriesnow.space/api/v0.1/countries/state/cities/q/q/q/q/q/q/q/q/q?

**

Thanks Jan

MartinsOnuoha commented 2 years ago

Hi Jan, thanks a bunch for reporting this, unfortunately we hadn't test with curl or other clients when we added this redirect. If this is the only endpoint you found this problem on do let me know, in the meantime I'm having a look at it right away.

jasteyn07 commented 2 years ago

Hello Martins

Thank you for your feedback and assistance.

Below are the endpoints I am struggling with. What I think is important it is only when Using post statements to the endpoint that these errors occur, when providing parameters for the search. As mentioned previously when I do this in Postman, I get the expected results, but from Bubble.io I get an error of Max Redirects. Bubble.io Support team has also looked in to this and responded with feedback further below. Furthermore, I have also tested the endpoints today and still receive the same error responses.

EndPoints: https://countriesnow.space/api/v0.1/countries/state/cities https://countriesnow.space/api/v0.1/countries/cities

From Bubble.io Support: I'm running some tests on our end now and still digging to see if I can find a cause of this behavior. Again, I can assure there have been no changes made recently to the maxRedirects settings of the API connector. I also played around with the settings in postman and took the maxRedirects allowed there all the way down to 1, and the call succeeded in postman, which is certainly strange.

Have you heard back from the API author yet? I'm very curious to hear if they have any insight to this behavior.

Thank you for looking into this. Best Regards Jan

MartinsOnuoha commented 2 years ago

Hi Jan,

The change definitely came from our end, for best practices we needed to convert make our post request to idempotent but also allow for backward compatibility. so here's what I'll do, I will revert this change and move it to feature/fix branch where we can investigate this problem internally, using Bubble as a point of reference (because really this is the only point where we've seen this problem). Then let me know if the problem still exists.

MartinsOnuoha commented 2 years ago

@TimAagaard Do you have any suggestions about resolving this issue?

TimAagaard commented 2 years ago

@MartinsOnuoha not sure of the solution but this is reproducible using curl. Everything works fine in postman and the unit tests.

Command line version to reproduce is the same as @jasteyn07 first post except add -L --max-redirs 2 to it. I've ruled out API caching as a culprit and disabling the cookie jar in postman didn't cause the issue so assuming postmans cookie jar setting is working, it's not cookie related.

Further investigation needed.

It's almost certainly a difference in how curl handles redirects versus node/express.

Judging by @jasteyn07 output the 9 /q's it looks like it's following the redirect but keeping the method as POST...I might be able to solve it by explicitly setting the method - I originally had that in the code but took it out because it made logging show the initial request as GET instead of POST.

Will follow up shortly.

TimAagaard commented 2 years ago

@MartinsOnuoha Good news - this isn't a bug...at-least not in our API. It might be a bug in Bubble.io if they are using curl to make requests.

The problem here is the --request POST switch given to curl. Remove that and it works fine. That switch is preventing curl from switching the method to GET. Verbose output, with that switch shows that it knows it should switch method due to the 301 redirect, but still continues to call the new route via POST.

* Switch from POST to GET
* Found bundle for host localhost: 0x244bb93dde0 [serially]
* Can not multiplex, even if we wanted to!
* Re-using existing connection! (#0) with host localhost
* Connected to localhost (127.0.0.1) port 3000 (#0)
> POST /api/v0.1/countries/state/cities/q?country=south%20africa&state=gauteng HTTP/1.1

Remove the --request POST from the curl command and it works as expected.

@jasteyn07 Note the above response. If Bubble.io is using curl for POST requests they should not be setting the --request POST switch, especially not if --data-raw is present as that implicitly causes the request to be POST.

@MartinsOnuoha I would mark this issue closed, as it is not a bug in the API, it is expected behavior from curl when using the --request POST switch.

MartinsOnuoha commented 2 years ago

@TimAagaard thanks for the thorough investigation, @jasteyn07 as this is mostly a curl configuration problem on Bubble's end I believe you can send @TimAagaard detailed response back to the Bubble team? Perhaps this would help them with resolving the problem you're facing. For now, however, we'll close this issue, feel free to open another if necessary.

jasteyn07 commented 2 years ago

Hi Martins,

Thank you for your time in investigating this. I will send this information to Bubble to have a look in seeing if they can fix this.

Best regards