curlconverter / curlconverter

Transpile curl commands into Python, JavaScript and 27 other languages
https://curlconverter.com
MIT License
7.17k stars 867 forks source link

Improve URL parameter handling #597

Closed Memexurer closed 6 months ago

Memexurer commented 7 months ago

When copying as curl from chrome, some characters aren't properly encoded (like asterisk). Then these characters make the whole param parsing fail. Example command: curl 'https://example.com?resolution=1600*900'

verhovsky commented 6 months ago

If you're talking about Python, this is intentional. If we change your command to

curl 'http://localhost:8888?resolution=1600900'

Then we can see what data it sends by running a simple netcat server with

nc -kl 8888

The curl command sends this data

GET /?resolution=1600*900 HTTP/1.1
Host: localhost:8888
User-Agent: curl/8.4.0
Accept: */*

The generated Python code for that command is

import requests

response = requests.get('http://localhost:8888?resolution=1600*900')

Which is correct because it sends the same query string

GET /?resolution=1600*900 HTTP/1.1
Host: localhost:8888
User-Agent: python-requests/2.31.0
Accept-Encoding: gzip, deflate
Accept: */*
Connection: keep-alive

If you're saying it should generate this code instead

import requests

params = {
    'resolution': '1600*900',
}

response = requests.get('http://localhost:8888', params=params)

then that wouldn't be correct because Requests does percent-encoding in that case and sends this data:

GET /?resolution=1600%2A900 HTTP/1.1

copying as curl from chrome, some characters aren't properly encoded

Is this really true? The browser is sending resolution=1600%2A900 but when you copy as curl it generates a command that sends resolution=1600*900? That would be an issue to fix in "Copy as cURL".

Memexurer commented 6 months ago
def uhhh(url):
    parse_result = urlparse(url)

    dict_result = parse_qs(parse_result.query, keep_blank_values=True)

    for key in dict_result:
        dict_result[key] = dict_result[key][0]

    pp = pprint.PrettyPrinter(indent=4,  sort_dicts=False)
    pp.pprint(dict_result)

anyways, here's a python code for generating that query params manually might help if someone finds this issue...