GIScience / ohsome-api

API for analysing OpenStreetMap history data
https://api.ohsome.org
GNU Affero General Public License v3.0
47 stars 8 forks source link

Swagger UI doesn't encode ! #19

Closed joker234 closed 4 years ago

joker234 commented 4 years ago

If I use building=yes and building!=no and type:node as filter parameter, swagger-ui generates this cURL request:

curl -X GET "https://api.ohsome.org/v1/elements/count?bboxes=8.625%2C49.3711%2C8.7334%2C49.4397&filter=building%3Dyes%20and%20building!%3Dno%20and%20type%3Anode&format=json&showMetadata=false&time=2014-01-01%2F2017-01-01%2FP1Y" -H  "accept: application/json"

To be executable in zsh and bash (and probably most other *NIX shells), the ! should be encoded as %21 according to https://en.wikipedia.org/wiki/Percent-encoding#Percent-encoding_reserved_characters.

What (in theory) also works is to not encode the =, so it would be …and%20building!=no%20and…, but I wouldn't recommend that.

FabiKo117 commented 4 years ago

This is also true for the request URL that is contained in the metadata of the response, see e.g. this example:

"metadata": {
    "executionTime": 1327,
    "description": "Total count of items in absolute values.",
    "requestUrl": "https://api.ohsome.org/v1/elements/count?bboxes=8.625%2C49.3711%2C8.7334%2C49.4397&filter=type%3Away%20and%20building%20%3D%20*%20and%20building%20!%3D%20no&format=json&showMetadata=yes&time=2014-01-01"
  }

So somewhere the '!' is somehow not encoded, whereas the rest (e.g. space as %20, equals as %3D, etc.) is encoded...

joker234 commented 4 years ago

This is also true for the request URL that is contained in the metadata of the response […]

The response always includes the received URL. No additional urlencode is applied. See:

% curl -X GET 'https://api.ohsome.org/v1/elements/count?bboxes=8.625,49.3711,8.7334,49.4397&filter=building=yes%20and%20type:node&showMetadata=true'
{
  "attribution" : {
    "url" : "https://ohsome.org/copyrights",
    "text" : "© OpenStreetMap contributors"
  },
  "apiVersion" : "1.0.0",
  "metadata" : {
    "executionTime" : 1062,
    "description" : "Total count of items in absolute values.",
    "requestUrl" : "https://api.ohsome.org/v1/elements/count?bboxes=8.625,49.3711,8.7334,49.4397&filter=building=yes%20and%20type:node&showMetadata=true"
  },
  "result" : [ {
    "timestamp" : "2020-06-29T03:00:00Z",
    "value" : 10.0
  } ]
}
FabiKo117 commented 4 years ago

Yeah, so you basically get the URL back that you've initially sent with the exception of spaces " " that are always encoded into "%20". Otherwise it would break the URL, so I guess that makes sense to encode them at least.

About the behavior of the swagger cURL suggestion: That seems weird that it encodes some signs and others not. The "*" sign for example, or the "(" ")" bracket signs are also not encoded, although they are on that list that you linked.

So this issue can only arise when you try to use use a generated URL (e.g. from swagger, or getting it from someone else) in your cURL request as a GET request for your mentioned shells, right? Because when using other access points (browser, postman) or POST requests, it either works or you have to encode the parameters yourself anyway.

joker234 commented 4 years ago

About the behavior of the swagger cURL suggestion: That seems weird that it encodes some signs and others not. The "*" sign for example, or the "(" ")" bracket signs are also not encoded, although they are on that list that you linked.

Yes it is a little bit weird.

So this issue can only arise when you try to use use a generated URL (e.g. from swagger, or getting it from someone else) in your cURL request as a GET request for your mentioned shells, right? Because when using other access points (browser, postman) or POST requests, it either works or you have to encode the parameters yourself anyway.

Sure, if you use another tool to create your requests, or the proper attributes of cURL to encode your parameters the requests work. Just the generated cURL requests by swagger UI are concerned of this issue. If you want to get rid of swagger UI anyway I wouldn't use much time to fix this, and set a close & wontfix label :)

tyrasd commented 4 years ago

I guess this is really more an issue of the swagger UI itself, and not really we can influence. So I agree it's best to close this one.

As a workaround one could replace " with ' in the swagger-generated curl commands. :shrug:

About the behavior of the swagger cURL suggestion: That seems weird that it encodes some signs and others not.

Under the hood swagger probably just uses javascript's encodeURIComponent which (for whatever reason) explicitly does not escape the ! and some other characters.

joker234 commented 4 years ago

As a workaround one could replace " with ' in the swagger-generated curl commands. shrug

This would be a problem for all Windows users, afaik.

tyrasd commented 4 years ago

does windows' shell have issues with ! also? I wanted to say that we could suggest this workaround to users who have an issue with unescaped ! in the curl command copied from swagger…