Closed shulltronics closed 1 year ago
Can you read this comment of mine to another user on another thread, and try again? I read in your code that you parse the SSLproxy line, but I think you close the connection to sslproxy. Your listening program should keep sslproxy connections (on both sides) open until sslproxy decides to close them. Also, please see the sample lp in the sources.
@sonertari: Thank you for your help. After looking more closely at that issue, I was able to get a "kind of" working example. In this code, I asynchronously listen for new connections from SSLproxy, parse the SSLproxy line, and then start forwarding the data between the two SSLproxy ports (client -> server, and vice versa).
If the EOF character is received, I return from the coroutines, essentially ending the connection. (I'm not sure if this is the proper action to take or not).
This works somewhat, in the sense that I can load most webpages through my listener script; however, the loading bar doesn't always complete, and some applications don't work at all. The main issue my experiments have shown is that sometimes the SSLproxy line doesn't seem to be present in the first line from SSLproxy. Furthermore, it seems that sometimes the connections that come from SSLproxy immediately return the EOF character.
I have explored the lp
example, and see where the SSLproxy line is parsed in prototcp.c
. I believe I am attempting to parse the line properly, and can't understand why some of the connections coming from SSLproxy seem to close right away of not contain the SSLproxy line.
So I guess my questions now are:
\n
terminated line?Thanks again for your help, and here is my code in case it's helpful:
import asyncio
import socket
import re
# Address to listen on
LISTENER_ADDR = '127.0.0.1'
LISTENER_PORT = 10101
# Global state keeping variables TODO make this the PacketMaster class, keeping much more detailed track of things
num_connections = 0
sslproxy_ports = []
def parse_sslproxy_line(payload):
"""
Extract the SSLproxy line from a payload string
"""
# First check that this packet contains the SSLproxy line
try:
sslproxy_line_index = payload.index("SSLproxy: ")
except:
return None
string = payload[sslproxy_line_index+10:] # remove the identifier
# get the comma delimited fields
parts = re.split(',', string)
# these regexs match the ip addresses and ports
addr_regex = r"\[(.*?)\]"
port_regex = r":([0-9]*)"
sslproxy_addr = re.search(addr_regex, parts[0]).group(1)
sslproxy_port = re.search(port_regex, parts[0]).group(1)
src_addr = re.search(addr_regex, parts[1]).group(1)
src_port = re.search(port_regex, parts[1]).group(1)
dst_addr = re.search(addr_regex, parts[2]).group(1)
dst_port = re.search(port_regex, parts[2]).group(1)
# single letter encryption scheme
encryption = parts[3][0]
return (sslproxy_addr, int(sslproxy_port), src_addr, int(src_port), dst_addr, int(dst_port), encryption)
async def server_client_forward(to_client, from_server):
"""
This coroutine is called each time an SSLproxy line is received and parsed by `client_server_forward`
It is responsible for asyncronously getting server responses from SSLproxy, and forwarding them back
for the client to receive
"""
(conn_ip, conn_port) = to_client.get_extra_info('peername')
print(f" {conn_port}: started server_client_forward task")
while not from_server.at_eof():
# Get server->client data from SSLproxy, and send back
server_client_data = await from_server.readline()
to_client.write(server_client_data)
await to_client.drain()
print(f" {conn_port}: exiting server_client_forward!")
async def client_server_forward(from_client, _to_client):
"""
This coroutine is called when SSLproxy establishes a connection with this program
* The data going from client->server is read from `from_client`,
the first packet of that data should contain the SSLproxy line.
* After parsing the SSLproxy line, another bidirectional socket stream is established
with SSLproxy on the assigned port, assigned to `to_server` and `from_server`
* The client->server data is the written to `to_server`.
* The server->client response is then received from SSLproxy via `from_server`,
and forwarded on to `to_client`.
"""
# Get some identifying info about the connection
global num_connections, sslproxy_ports
(conn_ip, conn_port) = _to_client.get_extra_info('peername') # TODO would it be equivalent to use from_client?
print(f"Connection from SSLproxy port {conn_port}")
sslproxy_ports.append(conn_port)
num_connections += 1
print(f" {conn_port}: number of connections: {num_connections}")
# First, get the SSLproxy line and populate the upstream address
sslproxy_ip = None
sslproxy_port = None
data = await from_client.readline()
try:
message = data.decode()
except Exception as e:
print(" ERROR decoding payload (returning):", str(e))
return
sslp_addrs = parse_sslproxy_line(message)
if sslp_addrs:
(sslp_ip, sslp_port, src_ip, src_port, dst_ip, dst_port, ssl) = sslp_addrs
print(f" {conn_port}: found SSLproxy line => {sslp_ip}:{sslp_port}")
print(f" {conn_port}: src address => {src_ip}:{src_port}")
print(f" {conn_port}: dst address => {dst_ip}:{dst_port}")
print(f" {conn_port}: original encryption: {ssl}")
sslproxy_ip = sslp_ip
sslproxy_port = sslp_port
### At this point, we have the port of where to send/receive data to/from the server,
# but we need this to also run asyncronously.
# Create an async connection to SSLproxy as specified in the SSLproxy line
try:
print(f" {conn_port}: connecting to SSLproxy at {sslproxy_ip}:{sslproxy_port}")
# TODO sometimes I have to sway the following two lines for the connections to work??
(from_server, to_server) = await asyncio.open_connection(sslproxy_ip, sslproxy_port)
#(from_server, to_server) = await asyncio.open_connection(sslproxy_ip, sslproxy_port, local_addr=(sslproxy_ip, sslproxy_port))
# Schedule the server_client_forward task, passing the relevant connections over
loop = asyncio.get_running_loop()
loop.create_task(server_client_forward(_to_client, from_server)) # the ONLY spot we use _to_client in this coro
except Exception as e:
print(f" {conn_port}: ERROR: couldn't connect back to SSLproxy:", str(e))
return
# Send this packet back to SSLproxy
#print(" {0}: [pre-loop] writing {1} bytes to to_server.".format(conn_port, len(data)))
to_server.write(data)
await to_server.drain()
# Now forward all subsequent packets to the open connection, in both directions
while not from_client.at_eof():
# Get client->server data from SSLproxy, and send back
client_server_data = await from_client.readline()
to_server.write(client_server_data)
await to_server.drain()
print(f" {conn_port}: task_a at EOF, exiting loop!")
async def main():
server = await asyncio.start_server(
client_server_forward, LISTENER_ADDR, LISTENER_PORT)
addrs = ', '.join(str(sock.getsockname()) for sock in server.sockets)
print(f'Serving on {addrs}')
async with server:
await server.serve_forever()
asyncio.run(main())
Upon further investigation, I found something interesting. The connections that are opened to my listener script that do not contain the SSLproxy line seem to correspond to BEV_EVENT_ERROR
lines in the debug logs of SSLproxy.
I have read a couple of other issues that mention this error, but they didn't seem super relevant to me until now. I am using certificates on my proxy machine that I generated with openssl
, and copied/installed the certificate on my client machine. Seeing as some of the ssl connections succeed, it doesn't seem like an issue with the ssl certificates. Something that I had read earlier made me set the VerifyPeer
option in the config to no
, but I can't seem to find again where I discovered that.
I've attached the debug log from SSLproxy (report.txt
), and the logs from my two listener scripts (identical scripts, just listening on different ports, one for tcp protocol, and the other for ssl protocol). From what I can tell, the times when the listener script cannot parse an SSLproxy line seem to correspong to the BEV_EVENT_ERRORs in the SSLproxy debug.
Can you see any reason for the errors? Let me know of any other experiments/details you need me to provide. Thanks again (in advance) for helping me out.
As mentioned in README, "SSLproxy inserts in the first packet the address and port it is expecting to receive the packets back from the program." So, sslproxy does not insert any further SSLproxy lines in any other packets. So, your program should maintain the state of each connection, and use the info in the SSLproxy line in the very first packet. I think the issues you observe are related with that.
Thanks for getting back with me. I'm not saying that my listener program is flawless, but I think the issues I'm having are more related to the BEV_EVENT_ERRORs I'm seeing. I'm putting the link to this file here just for reference. I will follow up if I discover anything that helps me solve this.
I've migrated my project over to using mitmproxy, which I think is a better solution for me right now. Thanks again, for your great work and willingness to help me!
Hi! Thanks for making this awesome tool. I'm using it for a university project, exploring the security of phone app communications. I've been reading though all of the documentation and running a lot of experiments, but am getting stuck with my listener program, which for now I just want to be a simple packet pass-through.
For my setup, I have a laptop forming a hotspot on one network interface, and connected to the internet via another. I am doing internet sharing by configuring some
iptables
rules (see iptables-setup.txt if interested) I haveSSLproxy
sitting between, processing the packets.I have been able to get
sslsplit
to work, as well asSSLproxy
in passthrough mode, but cannot seem to get my listener program to properly communicate with SSLproxy.I'm starting SSLproxy with
sslproxy -l connections.log -j tmp/sslproxy/ -S sslproxy-logs/ -X sslproxy.pcap -k ca.pem -c ca.crt -f sslproxy.conf
, and my config file is as follows:My listener script is as follows:
After starting my program and SSLproxy, and connecting my phone to generate some traffic, the program outputs are as follows: listener-output.txt sslproxy-output.txt
I know this is a lot to read, but I would really appreciate any guidance you could give me. I have done a lot to try to get this working and I think I just need a little help. Thanks so much!!