search

bogdanfinn / tls-client

net/http.Client like HTTP Client with options to select specific client TLS Fingerprints to use for requests.
BSD 4-Clause "Original" or "Old" License
668 stars 133 forks source link

[Bug]: Need to follow Cookies' domain rules #75

Open hzhan147 opened 9 months ago

hzhan147 commented 9 months ago

TLS client version

v1.6.0

System information

Windows 10, Linux Ubuntu

Issue description

Version: 1.6.0

I am using Python_TLS_Client and found this bug.

Python codes that used this project for requests: `session = tls_client.Session( client_identifier="chrome107", random_tls_extension_order=True )

session.get('https://httpbin.org/cookies/set/testcookie/12345') session.get('https://httpbin.org/cookies/set/abc/67890') print(session.cookies.get_dict()) session.cookies.set("test123", "test", domain="example.org") res = session.get("https://httpbin.org/headers", proxy="http://localhost:8888") print(res.text)`

Results from Fiddler inspection: `Request sent 41 bytes of Cookie data:

testcookie=12345
abc=67890
test123=test

`

The correct result should be:

`Request sent X bytes of Cookie data:

testcookie=12345
abc=67890

`

The 'test123' cookie shouldn't be included in the cookies as the domain was different.

Steps to reproduce / Code Sample

  1. Set a cookie without domain
  2. Send a request and verify the cookie was in the request
  3. Set a cookie with a domain that IS different than the url
  4. Send a request and verify the first cookie was in the request, but not the second one as domain was different.
bogdanfinn commented 9 months ago

@hzhan147 here is a plain python implementation of your use case without using the tls_client package your are mentioning.

TLDR: seem like you want to open that issue here: https://github.com/FlorianREGAZ/Python-Tls-Client/issues

import ctypes
import json

# load the tls-client shared package for your OS you are currently running your python script (i'm running on mac)
library = ctypes.cdll.LoadLibrary('./../dist/tls-client-xgo-1.6.0-linux-amd64.so')

# extract the exposed request function from the shared package
request = library.request
request.argtypes = [ctypes.c_char_p]
request.restype = ctypes.c_char_p

addCookiesToSession = library.addCookiesToSession
addCookiesToSession.argtypes = [ctypes.c_char_p]
addCookiesToSession.restype = ctypes.c_char_p

requestPayload = {
    "tlsClientIdentifier": "chrome_105",
    "sessionId": "my-session-id",
    "followRedirects": True,
    "headers": {
        "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
        "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36",
        "accept-encoding": "gzip, deflate, br",
        "accept-language": "de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7"
    },
    "headerOrder": [
        "accept",
        "user-agent",
        "accept-encoding",
        "accept-language"
    ],
    "requestUrl": "https://httpbin.org/cookies/set/testcookie/12345",
    "requestMethod": "GET",
}

# this is a pointer to the response
response = request(json.dumps(requestPayload).encode('utf-8'))

# we dereference the pointer to a byte array
response_bytes = ctypes.string_at(response)

# convert our byte array to a string (tls client returns json)
response_string = response_bytes.decode('utf-8')

# convert response string to json
response_object = json.loads(response_string)

# print out output
print(response_object)

requestPayload = {
    "tlsClientIdentifier": "chrome_105",
    "sessionId": "my-session-id",
    "followRedirects": True,
    "headers": {
        "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
        "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36",
        "accept-encoding": "gzip, deflate, br",
        "accept-language": "de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7"
    },
    "headerOrder": [
        "accept",
        "user-agent",
        "accept-encoding",
        "accept-language"
    ],
    "requestUrl": "https://httpbin.org/cookies/set/abc/67890",
    "requestMethod": "GET",
}

# this is a pointer to the response
response = request(json.dumps(requestPayload).encode('utf-8'))

# we dereference the pointer to a byte array
response_bytes = ctypes.string_at(response)

# convert our byte array to a string (tls client returns json)
response_string = response_bytes.decode('utf-8')

# convert response string to json
response_object = json.loads(response_string)

# print out output
print(response_object)

cookiePayload = {
    "sessionId": "my-session-id",
    "url": "https://example.org",
    "cookies": [{
        "name": "test123",
        "value": "test",
        "domain": "example.org",
    }]
}

cookieResponse = addCookiesToSession(json.dumps(cookiePayload).encode('utf-8'))
# we dereference the pointer to a byte array
cookieResponse_bytes = ctypes.string_at(cookieResponse)
# convert our byte array to a string (tls client returns json)
cookieResponse_string = cookieResponse_bytes.decode('utf-8')
# convert response string to json
cookieResponse_object = json.loads(cookieResponse_string)

# print out output
print(cookieResponse_object)

requestPayload = {
    "tlsClientIdentifier": "chrome_105",
    "sessionId": "my-session-id",
    "followRedirects": True,
    "headers": {
        "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
        "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36",
        "accept-encoding": "gzip, deflate, br",
        "accept-language": "de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7"
    },
    "headerOrder": [
        "accept",
        "user-agent",
        "accept-encoding",
        "accept-language"
    ],
    "requestUrl": "https://httpbin.org/headers",
    "requestMethod": "GET",
}

# this is a pointer to the response
response = request(json.dumps(requestPayload).encode('utf-8'))

# we dereference the pointer to a byte array
response_bytes = ctypes.string_at(response)

# convert our byte array to a string (tls client returns json)
response_string = response_bytes.decode('utf-8')

# convert response string to json
response_object = json.loads(response_string)

# print out output
print(response_object)

requestPayload = {
    "tlsClientIdentifier": "chrome_105",
    "sessionId": "my-session-id",
    "followRedirects": True,
    "headers": {
        "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",
        "user-agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/105.0.0.0 Safari/537.36",
        "accept-encoding": "gzip, deflate, br",
        "accept-language": "de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7"
    },
    "headerOrder": [
        "accept",
        "user-agent",
        "accept-encoding",
        "accept-language"
    ],
    "requestUrl": "https://example.org/headers",
    "requestMethod": "GET",
}

# this is a pointer to the response
response = request(json.dumps(requestPayload).encode('utf-8'))

# we dereference the pointer to a byte array
response_bytes = ctypes.string_at(response)

# convert our byte array to a string (tls client returns json)
response_string = response_bytes.decode('utf-8')

# convert response string to json
response_object = json.loads(response_string)

# print out output
print(response_object)