correct curl is not translated to Python code

[beniev]

I get a parsing error with this curl command:

curl "https://www.nadlan.gov.il/KeshetAnnan.REST/Main/GetDataByQuery?query=^%^D7^%^A9^%^D7^%^9B^%^D7^%^95^%^D7^%^A0^%^D7^%^AA^%^20^%^D7^%^A8^%^D7^%^9E^%^D7^%^AA^%^20^%^D7^%^90^%^D7^%^A9^%^D7^%^9B^%^D7^%^95^%^D7^%^9C^%^2C^%^20^%^D7^%^99^%^D7^%^A8^%^D7^%^95^%^D7^%^A9^%^D7^%^9C^%^D7^%^99^%^D7^%^9D^%^20" -H "Accept-Encoding: gzip, deflate, br" -H "Accept-Language: en-US,en;q=0.8" -H "User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36" -H "Accept: application/json, text/plain, */*" -H "Referer: https://www.nadlan.gov.il/^%^D7^%^A8^%^D7^%^97^%^D7^%^95^%^D7^%^91^%^20^%^D7^%^99^%^D7^%^9D^%^20^%^D7^%^A1^%^D7^%^95^%^D7^%^A3^%^208,^%^20^%^D7^%^99^%^D7^%^A8^%^D7^%^95^%^D7^%^A9^%^D7^%^9C^%^D7^%^99^%^D7^%^9D^%^20" -H "Cookie: _ga=GA1.3.584052889.1509031917; _gid=GA1.3.1552049848.1509031917; keshet-client-token=eL7onXBKTzKmYReUJ2zri2jG0hm0O7f8I0eNK+2Dzby27SrBUoI7pNz2NbE+gMyeYaLHhH/NBlctm0OyaqgSe2P17fB5BCG97LUfhr/PGZaoOqzBAxtHtWPfGYj1Bp2yGiWCAJQWHNsetxsjkOJ49exg5whBmzwRRe2Z+t7l5ygQaeE2C5easc3PETUgTm5m" -H "Connection: keep-alive" --compressed

[NickCarneiro]

Debugging info for me: Chrome stack trace

URIError: URI malformed
    at decodeURIComponent (<anonymous>)
    at Object.e.exports (main.js:2)
    at Object.c [as parseCurlCommand] (main.js:2)
    at Object.f [as toPython] (main.js:1)
    at HTMLTextAreaElement.a (main.js:2)

Reproduce exception:

decodeURIComponent('query=%5E%%5ED7%5E%%5EA9%5E%%5ED7%5E%%5E9B%5E%%5ED7%5E%%5E95%5E%%5ED7%5E%%5EA0%5E%%5ED7%5E%%5EAA%5E%%5E20%5E%%5ED7%5E%%5EA8%5E%%5ED7%5E%%5E9E%5E%%5ED7%5E%%5EAA%5E%%5E20%5E%%5ED7%5E%%5E90%5E%%5ED7%5E%%5EA9%5E%%5ED7%5E%%5E9B%5E%%5ED7%5E%%5E95%5E%%5ED7%5E%%5E9C%5E%%5E2C%5E%%5E20%5E%%5ED7%5E%%5E99%5E%%5ED7%5E%%5EA8%5E%%5ED7%5E%%5E95%5E%%5ED7%5E%%5EA9%5E%%5ED7%5E%%5E9C%5E%%5ED7%5E%%5E99%5E%%5ED7%5E%%5E9D%5E%%5E20')
URIError: malformed URI sequence [Learn More]

Curl payload for this command:

nick@zilker:~/development/curltorequests$ nc -l 3001
GET /GetDataByQuery?query=^%^D7^%^A9^%^D7^%^9B^%^D7^%^95^%^D7^%^A0^%^D7^%^AA^%^20^%^D7^%^A8^%^D7^%^9E^%^D7^%^AA^%^20^%^D7^%^90^%^D7^%^A9^%^D7^%^9B^%^D7^%^95^%^D7^%^9C^%^2C^%^20^%^D7^%^99^%^D7^%^A8^%^D7^%^95^%^D7^%^A9^%^D7^%^9C^%^D7^%^99^%^D7^%^9D^%^20 HTTP/1.1
Host: localhost:3001
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.8
User-Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/61.0.3163.100 Safari/537.36
Accept: application/json, text/plain, /
Referer: https://www.nadlan.gov.il/^%^D7^%^A8^%^D7^%^97^%^D7^%^95^%^D7^%^91^%^20^%^D7^%^99^%^D7^%^9D^%^20^%^D7^%^A1^%^D7^%^95^%^D7^%^A3^%^208,^%^20^%^D7^%^99^%^D7^%^A8^%^D7^%^95^%^D7^%^A9^%^D7^%^9C^%^D7^%^99^%^D7^%^9D^%^20
Cookie: _ga=GA1.3.584052889.1509031917; _gid=GA1.3.1552049848.1509031917; keshet-client-token=eL7onXBKTzKmYReUJ2zri2jG0hm0O7f8I0eNK+2Dzby27SrBUoI7pNz2NbE+gMyeYaLHhH/NBlctm0OyaqgSe2P17fB5BCG97LUfhr/PGZaoOqzBAxtHtWPfGYj1Bp2yGiWCAJQWHNsetxsjkOJ49exg5whBmzwRRe2Z+t7l5ygQaeE2C5easc3PETUgTm5m
Connection: keep-alive

Calling querystring.parse in the browser makes a call to decodeURIComponent which throws that exception when it encounters an error. I assume that curl is more tolerant or doesn't try to parse the parameters. Gotta dig into it.

Edit: I guess we don't really need to parse query strings at all for GET requests. We can just preserve the url.

[nicktimko]

I like parsing query strings as it helps when the true output is for humans (i.e. for reverse engineering something). I have a notion why the caveat is needed because of possible differences, and if parsing fails, just fall back on literally copying the querystring into the URL, but +1 for parsing if possible!

[romadrutskii]

PR #233 fixed the issue.

[verhovsky]

Duplicate of #143