jolav / codetabs

Free Online Services. Github/GitLab star history. Count Lines of Code. CORS proxy server. IP GeoLocation. HTTP Headers. Random Data. Api weather temp. Alexa ranking.
https://codetabs.com
BSD 3-Clause "New" or "Revised" License
231 stars 30 forks source link

Query parameters don't work properly #20

Open GitBoudewijn opened 2 years ago

GitBoudewijn commented 2 years ago

Hi,

Thanks for your great service, but I've come across an issue:

Normally when you set query parameters in an url you use something like encodeURIComponent(), especially when you have another url as the value which can contain special characters like ?, = and &. However with your service when you do this:

var url = 'https://api.codetabs.com/v1/proxy/?quest=' + encodeURIComponent('http://example.com');

Requesting that url gives this response:

{
 "Error": "http://http%3A%2F%2Fexample.com is not a valid resource"
}

So apparently the server doesn't actually treat it as a query string. Either it has to decode the query string, or you could allow urls like 'https://api.codetabs.com/v1/proxy/http://example.com' (in which case you would use encodeURI() instead of encodeURIComponent()).

Tsuk1ko commented 1 year ago

Hi,

Thanks for your great service, I have a similar issue.

For example I need to request http://example.com/?a=1&b=2. I make a request like

https://api.codetabs.com/v1/proxy/?quest=http://example.com/?a=1&b=2

But the request url received by the server is http://example.com/?a=1%26b=2, the & was encoded to %26.

And as @GitBoudewijn said, I can't make a request like

https://api.codetabs.com/v1/proxy/?quest=http%3A%2F%2Fexample.com%2F%3Fa%3D1%26b%3D2

The API doesn't do decodeURIComponent().

jolav commented 1 year ago

At this point the server correctly understands the urls sent with encodeURIComponent() (at least i hope so)

This is a valid request var url = 'https://api.codetabs.com/v1/proxy/?quest=' + encodeURIComponent('http://example.com'); This one too https://api.codetabs.com/v1/proxy/?quest=http%3A%2F%2Fexample.com%2F%3Fa%3D1%26b%3D2

I think the problem is that when the server makes the request to the third party it does not reconstruct the new request building a complete URL by encoding individual parts.

Fixing that would give a lot of power to the proxy api and that's something I'm not sure I want to happen. As it says on the landing page this is a CORS proxy to bypass same-origin policy related to AJAX requests to third party services allowing access resources from any website. You can use to prevent mixed content of images and JSON data proxying the resources to serve them under https. The original idea was to gather images(png, jpeg, ...), files (json, xml, txt , pdf ...)

As of today there are about 5 million proxy requests per day. Last year peaks of 40 million were reached. There are problems with misuse of api for phishing, brute force attacks and all kinds of abuses. Because of this I have set up a banning system.

Proyect is very nice but I don't want to spend all day checking that no one abuses.

Unless you tell me otherwise i will change the label from bug to enhancement and then i will think about it.

I am very thankful for the feedback

Tsuk1ko commented 1 year ago

Ok, I understand your idea, preventing abuse is a hard problem.

I think it's fine to keep the existing logic.

Thank you for your reply.