kpdyer / fteproxy

programmable proxy for censorship circumvention
https://fteproxy.org/
Apache License 2.0
149 stars 21 forks source link

--upstream-format XXX --downstream-format YYY #164

Closed Stepman123 closed 6 years ago

Stepman123 commented 10 years ago

How can I change the http protocol to another torrent TCP? ISP does not block tоrrent tср.

kpdyer commented 10 years ago

Thanks for reporting an issue.

Can you be a bit more specific? What are you trying to do and what [unexpected] behavior are you experiencing?

Stepman123 commented 10 years ago

Sorry for my English. My ISP uses throttling (Shaping 16-64 kbyt/s) for all traffic except torrent TCP. I tested it as a torrent stream (torrent TV), uTorrent and special online test. Torrent gets a speed of up to 5 Mbyt/s even at 1 connect. So I need a torrent tunnel.

kpdyer commented 10 years ago

Your ISP throttles all non-torrent traffic? That's a very specific policy. Are you sure? Out of curiosity have you tried other things, such as non-HTTP traffic, encrypted traffic, etc.?

Can you try each of the four default formats provided with fteproxy? If none of them work, I'll investigate adding a torrent protocol.

Stepman123 commented 10 years ago

Your ISP throttles all non-torrent traffic? Yes.

That's a very specific policy. Are you sure? Yes

Out of curiosity have you tried other things, such as non-HTTP traffic, encrypted traffic, etc.? I tried to VPN (TCP, UDP), ICMP, various http (compressed and without compression), https, rdp, etc.

Can you try each of the four default formats provided with fteproxy? At first I tried it with TOR, but to no avail. Now I installed fteproxy a remote Windows Serwer 2003. What arguments is needed for command-line --upstream-format?

kpdyer commented 10 years ago

fteproxy ships with three pre-defined formats: http, smb, ssh, and "dummy". To change a format, you only have to specify the format client-side, it's automatically negotiated in-band.

To use HTTP:

fteproxy --mode client --upstream-format manual-http-request --downstream-format manual-http-response

Similarly, to use ssh:

fteproxy --mode client --upstream-format manual-ssh-request --downstream-format manual-ssh-response

Or, to use the "dummy" format, which means that all bytes on the wire are unformatted:

fteproxy --mode client --upstream-format manual-dummy-request --downstream-format manual-dummy-response
Stepman123 commented 10 years ago

When I run the client and server on the same machine, it runs fine, but can not establish a connection between the client and the server. Run as usual --mode client --mode server Sniffer showed that the connection is established on port 8080, but the server did not respond to a GET request twice. For the test I used Proxifier test; SOCKS5 127.0.0.1:8079

kpdyer commented 10 years ago

Are you setting your client_ip, server_ip, and proxy_ip parameters correctly?

Stepman123 commented 10 years ago

I run without parameters. Should work?

kpdyer commented 10 years ago

By default fteproxy binds to localhost, 127.0.0.1. Hence, if the client and server are on different hosts you'll have to specify the server_ip client-side and server-side. Otherwise, the client won't know where to find the server and the server won't know what interface to bind to.

Stepman123 commented 10 years ago

This is understandable. I run the server and client on the same computer. ?

kpdyer commented 10 years ago

Hmm. I guess I don't understand the problem, then. Can you explain (1) what you're trying to do and (2) how fteproxy is not behaving as expected?

Stepman123 commented 10 years ago

1) I am trying to build a chain of socks5client > fteproxy client > fteproxy server > internet.

2) fteproxy server does not respond to a GET request. 127.0.0.1:8080. This can be seen by using a sniffer.

kpdyer commented 10 years ago

In order for this to work, you would need to have a SOCKS server running on 127.0.0.1:8081.

Stepman123 commented 10 years ago

Now everything has turned out. Also with the server. But need your help, that would make a torrent protocol.

kpdyer commented 10 years ago

Hmm.

Would you expect this to be a torrent protocol over UDP?

Stepman123 commented 10 years ago

Torrent over UDP - throttling 64 kbps.

kpdyer commented 10 years ago

So, are you saying you need a UDP-based Torrent format?

Unfortunately, for now, fteproxy only supports TCP-based formats.

Stepman123 commented 10 years ago

No, I need a TCP-based formats.

kpdyer commented 10 years ago

Isn't BitTorrent traffic predominantly UDP?

Stepman123 commented 10 years ago

In my uTorrent UDP is disabled, because the speed is limited as I wrote - 64 kbps. For TCP torrent speed is not limited. There is Torrent Stream TV, which runs over TCP. It is also not limited.

kpdyer commented 10 years ago

Ok, give the following a try. Use fteproxy 0.2.17 and update your fteproxy/defs/20131224.json to the following, for both your client and server:

{
    "bittorrent-request": {
        "regex": "^(\\x13BitTorrent protocol|GET /announce\\?(info_hash|peer_id|ip|port|uploaded|downloaded|left|event)).*$"
    },
    "bittorrent-response": {
        "regex": "^(\\x13BitTorrent protocol|GET /announce\\?(info_hash|peer_id|ip|port|uploaded|downloaded|left|event)).*$"
    }
}

Then, start your client as

fteproxy --mode client --upstream-format bittorrent-request --downstream-format bittorrent-response --server_ip X --server_port Y

Let me know if that works for you.

Stepman123 commented 10 years ago

I used version 0.2.18. (0.2.17 not found) Startup error:

C:\Windows\system32>C:\Users\Witns\Downloads\fteproxy\fteproxy.exe --mode clie nt --upstream-format bittorrent-request --downstream-format bittorrent-response fteproxy Copyright (C) 2012-2014 Kevin P. Dyer This program comes with ABSOLUTELY NO WARRANTY. This is free software, and you are welcome to redistribute it under certain conditions.

Traceback (most recent call last): File "fteproxy\cli.pyc", line 150, in run File "fteproxy\regex2dfa.pyc", line 30738, in regex2dfa Exception: Regex not supported, please see fte/regex2dfa.py. ERROR: FTEMain terminated unexpectedly: Regex not supported, please see fte/rege x2dfa.py. File "threading.pyc", line 781, in bootstrap File "threading.pyc", line 808, in __bootstrap_inner File "fteproxy\cli.pyc", line 198, in run File "fteproxy__init.pyc", line 41, in fatal_error

C:\Windows\system32>

kpdyer commented 10 years ago

Give 0.2.16 a try: https://fteproxy.org/dist/0.2.16/fteproxy-0.2.16-windows-i386.zip

Stepman123 commented 10 years ago

0.2.16 works, but speed is limited. The regex that you gave, it is similar to the announce-server connection, but need only peer to peer.

kpdyer commented 10 years ago

Is this closer to what you need?

http://l7-filter.sourceforge.net/layer7-protocols/protocols/bittorrent.pat

Stepman123 commented 10 years ago

I beg your pardon. dummy works just as I need! Thank you! There is one question only: How to reduce CPU usage from fteproxy.exe? Disable encryption?

kpdyer commented 10 years ago

Glad to hear it.

What problems are you having w.r.t. performance? Can you tell me a bit more about the specifications of your system?

Stepman123 commented 10 years ago

Celeron 2.2; RAM 512 CPU load of 10-100% 30 sec to update this page. I understand that it should be for an old computer, but is it possible to disable encryption?

kpdyer commented 10 years ago

No, not really possible to disable encryption. That would have a number of negative implications, anyways.

30 sec. is way too much to load this page.

A couple ideas: (1) Have you tried obfsproxy? https://gitweb.torproject.org/pluggable-transports/obfsproxy.git (2) Is your fteproxy server located geographically close (within reason) to your client?

Stepman123 commented 10 years ago

1) obfsproxy only works with Tor-browser. 2) The problem is on the provider side. When you create dozens of TCP connections is locked (timeout). I checked on another provider, there is no such problem.

Q: Is it possible to enable tunneling? That is a persistent TCP connection between fte-server and fte-client. Or is it better to have a choice of optional number of connections 1-2-4-8 and so on. In my case, the compound 4 was normal.

kpdyer commented 10 years ago

What are you using server-side for your proxy? (i.e. what's listening server-side on your proxy_ip:proxy_port address?)

Stepman123 commented 10 years ago

This HandyCache. But it does not matter how it seems to me, so other provider on the client-side it was no problem.

kpdyer commented 10 years ago

Can you try using an SSH tunnel, to determine if that resolves this issue?

|ssh client| -> |fteproxy client| <--- internet --> |fteproxy server| -> |server's ssh port|

Just use "ssh -D [port] fteproxy_client_ip:fteproxy_client_port" client-side, or putty, to effectively create a single fteproxy stream.

If performance dramatically increases for you in the configuration, then I'll raise an issue and investigate further. If I understand your existing configuration, you shouldn't be encountering any performance issues.

kpdyer commented 10 years ago

Stepman - Are you still having CPU issues? Or can I close this thread?