Modify proxy response for certain urls

[JavaProgswing]

I want to intercept the request responses and return a custom status code and response body for a list of urls. Can someone guide me in attempting this?

[JavaProgswing]

???

[JJ-Author]

approach is to write your own plugin (look at the hooks available here) we started to have a look at all the example plugins to figure things out. maybe you can get inspiration from our project since we also alter requests https://github.com/dbpedia/ontology-time-machine/blob/main/ontologytimemachine/custom_proxy.py unfortunately I have no resources (since I am just a newbie user myself) to further guide you, but I agree that documentation needs improvement.

NOTE: selective https interception seems broken in proxypy at the moment in case you would need to change https requests

[JavaProgswing]

Thanks, I'll check that out. When I was trying with https urls it works but I only seem to receive CONNECT request methods.

[JavaProgswing]

and whenever I try to edit the requests i get requests.exceptions.SSLError: HTTPSConnectionPool(host='google.com', port=443): Max retries exceeded with url: / (Caused by SSLError(SSLError(1, '[SSL: WRONG_VERSION_NUMBER] wrong version number (_ssl.c:1002)')))

[abhinavsingh]

We have following examples on how to modify request/response, does none of these help? Note that, when you modify body of the responses, you must also modify necessary headers as per new body e.g. content length, content type etc

https://github.com/abhinavsingh/proxy.py?tab=readme-ov-file#modifypostdataplugin https://github.com/abhinavsingh/proxy.py?tab=readme-ov-file#modifychunkresponseplugin https://github.com/abhinavsingh/proxy.py?tab=readme-ov-file#modifyrequestheaderplugin

[JavaProgswing]

I only receive CONNECT method when connecting to proxy with a url. how will I check if tls interception works?

[abhinavsingh]

If you see CONNECT only, it means TLS intercept is simply not working. Please make sure basic TLS interception examples are working for you before proceeding

[JavaProgswing]

The tls interception examples don't seem to work for me, i generated ca-key, ca-cert and ca-signing-key from wsl then ran the program on windows. but I only receive CONNECT method requests.

[JavaProgswing]

I'm getting the error [WinError 2] The system cannot find the file specified ssl.create_default_context(ssl.Purpose.SERVER_AUTH, cafile=ca_file) IN proxy\core\connection\server.py

when trying to connect with a https url.

[JJ-Author]

my advice is trying to get it running on wsl first. make sure that openssl is installed in that. Note that you need openssl for interception not only for generating the ca cert files, so the claim that proxypy has zero dependencies ist not valid (anymore).

as a general note: you never provide the startup commands, your code, and the full stack trace. this heavily decreases the chance that somebody can help you.

My guesstimate is that you dont have openssl installed or are passing the ca cert files wrong.

[JavaProgswing]

PROXY CODE: from proxy.http.proxy import HttpProxyBasePlugin from proxy.http.parser import HttpParser import proxy import sys

class RequestPlugin(HttpProxyBasePlugin): def init(self, *args, *kwargs): super().init(args, **kwargs)

def handle_client_request(self, request: HttpParser):
    print(f"Request _url: {request._url}")
    print(f"Request.method: {request.method}")
    print(f"Request protocol: {request.protocol}")
    print(f"Request host: {request.host}")
    print(f"Request path: {request.path}")

    print(f"Request properties: {vars(request)}")

    return request

if name == "main":

sys.argv += [
    "--ca-key-file",
    "..\\https-interception-proxypy-main\\ca-key.pem",
    "--ca-cert-file",
    "..\\https-interception-proxypy-main\\ca-cert.pem",
    "--ca-signing-key-file",
    "..\\https-interception-proxypy-main\\ca-signing-key.pem",
]
sys.argv += [
    "--hostname",
    "127.0.0.1",
    "--port",
    "8080",
    "--plugins",
    __name__ + ".RequestPlugin",
    "--log-level",
    "d",
]

proxy.main()

CLIENT CODE:

import requests proxy = { 'http': 'http://127.0.0.1:8080', 'https': 'https://127.0.0.1:8080' }

response = requests.get('https://google.com/', proxies=proxy) print(response.text) print(response.headers)

What could be the problem here?

[JavaProgswing]

it does not work for https urls.

[abhinavsingh]

I don't see your plugin trying to modify the response. Try to modify the response chunks , try this method https://github.com/abhinavsingh/proxy.py/blob/develop/proxy/http/proxy/plugin.py#L130 , see existing examples overriding this method. Lmk.

[JavaProgswing]

from proxy.http.proxy import HttpProxyBasePlugin
from proxy.common.utils import build_http_response
import proxy
import requests
from http.client import responses
import sys

class ProxyPlugin(HttpProxyBasePlugin):
    def handle_client_request(self, request):
        print("Handle client request hook")
        print(
            f"Request method: {request.method} - Request host: {request.host} - Request path: {request.path} - Request headers: {request.headers}"
        )
        mock_response = requests.Response()
        mock_response.status_code = 200
        mock_response.url = 'https://example.com/success'
        mock_response.headers['Content-Type'] = 'text/html'
        mock_response._content = b'<html><body><h1>To be implemented</h1></body></html>'
        self.queue_response(mock_response)
        return None

    def queue_response(self, response):
        self.client.queue(
            build_http_response(
                response.status_code,
                reason=bytes(responses[response.status_code], "utf-8"),
                headers={
                    b"Content-Type": bytes(
                        response.headers.get("Content-Type"), "utf-8"
                    )
                },
                body=response.content,
            )
        )

if __name__ == "__main__":
    sys.argv += [
        "--ca-key-file",
        "..\\Python\\SeleniumProxy\\ca-key.pem",
        "--ca-cert-file",
        "..\\Python\\SeleniumProxy\\ca-cert.pem",
        "--ca-signing-key-file",
        "..\\Python\\SeleniumProxy\\ca-signing-key.pem",
    ]
    sys.argv += [
        "--hostname",
        "127.0.0.1",
        "--port",
        "8080",
        "--plugins",
        __name__ + ".ProxyPlugin",
    ]

    proxy.main()

It doesn't work this way, curl returns

• Host localhost:8080 was resolved.
• IPv6: ::1
• IPv4: 127.0.0.1
• Trying [::1]:8080...
• Trying 127.0.0.1:8080...
• Connected to localhost (127.0.0.1) port 8080
• CONNECT tunnel: HTTP/1.1 negotiated
• allocate connect buffer
• Establish HTTP proxy tunnel to example.com:443

  CONNECT example.com:443 HTTP/1.1 Host: example.com:443 User-Agent: curl/8.7.1 Proxy-Connection: Keep-Alive

  < HTTP/1.1 200 OK < Content-Type: text/html < Content-Length: 52

• Ignoring Content-Length in CONNECT 200 response <
• CONNECT phase completed
• CONNECT tunnel established, response 200
• schannel: disabled automatic use of client certificate
• ALPN: curl offers http/1.1
• schannel: next InitializeSecurityContext failed: SEC_E_INVALID_TOKEN (0x80090308) - The token supplied to the function is invalid
• Closing connection
• schannel: shutting down SSL/TLS connection with example.com port 443 curl: (35) schannel: next InitializeSecurityContext failed: SEC_E_INVALID_TOKEN (0x80090308) - The token supplied to the function is invalid