msantos / procket

Erlang interface to low level socket operations
http://blog.listincomprehension.com/search/label/procket
BSD 3-Clause "New" or "Revised" License
283 stars 80 forks source link

Reading data from socket and send it again #24

Open amuessig opened 8 years ago

amuessig commented 8 years ago

Hey,

I am trying to build up a middlebox device which is reading packets from a socket, check some stuff like IP address and (if the check matches) send the packet out through the socket. Later, I will be migrating the socket, thus I need to use raw sockets. My code is as follow:

init({Parent, Port}) ->
    register(?MODULE,self()),
    proc_lib:init_ack(Parent, {ok, self()}),
    {ok, ListenSock} = procket:open(Port, [{protocol, 16#0008}, {type, raw}, {family, packet}]),
    P = erlang:open_port({fd, ListenSock, ListenSock}, [binary, stream]),
    accept(P).

How can I bind the opened socket to a interface? I tried this: {ok, ListenSock} = procket:open(Port, [{protocol, 16#0008}, {type, raw}, {family, packet}, {interface, "vnf-eth0"}]), but it didn't work.

Furthermore, trying the listening method, I am not sure how to write back the data..:

accept(ListenSock) ->
    case gen_tcp:accept(ListenSock) of
        {ok, Client} ->
            io:format("foo~n"),
            Tmp = rule_database:lookup_rule_table(Client),
            if Tmp ->
                %if the lookup is positive, send it back
                {ok, _} = procket:sendTo(ListenSock);
            true ->
                % otherwise don't send it
                gen_tcp:close(Client)
                %erlang:port_close(Client)
            end,
            accept(ListenSock);
        _Error ->
            ok
    end.

Any suggestions/help? Thanks!

msantos commented 8 years ago

On Thu, Jan 21, 2016 at 09:37:21AM -0800, amuessig wrote:

Hey,

I am trying to build up a middlebox device which is reading packets from a socket, check some stuff like IP address and (if the check matches) send the packet out through the socket Later, I will be migrating the socket, thus I need to use raw sockets My code is as follow:

init({Parent, Port}) ->
    register(?MODULE,self()),
    proc_lib:init_ack(Parent, {ok, self()}),
    {ok, ListenSock} = procket:open(Port, [{protocol, 16#0008}, {type, raw}, {family, packet}]),
    P = erlang:open_port({fd, ListenSock, ListenSock}, [binary, stream]),
    accept(P)

How can I bind the opened socket to a interface? I tried this: {ok, ListenSock} = procket:open(Port, [{protocol, 16#0008}, {type, raw}, {family, packet}, {interface, "vnf-eth0"}]), but it didn't work

procket calls setsockopt(SO_BINDTODEVICE) for the interface option. According to socket(7):

SO_BINDTODEVICE

    Note that this works only for  some socket types, particularly
    AF_INET sockets.  It is not supported for packet sockets (use normal
    bind(2) there).

So we need to use bind(2). There's a version of bind() in the procket PF_PACKET module that works with packet sockets:

{ok, ListenSock} = procket:open(Port, [{protocol, 16#0008}, {type, raw}, {family, packet}]),
IfIndex = packet:ifindex(ListenSock, "vnf-eth0"),
ok = packet:bind(ListenSock, IfIndex)

Furthermore, trying the listening method, I am not sure how to write back the data: accept(ListenSock) -> case gen_tcp:accept(ListenSock) of {ok, Client} -> io:format("foo~n"), Tmp = rule_database:lookup_ruletable(Client), if Tmp -> %if the lookup is positive, send it back {ok, } = procket:sendTo(ListenSock); true -> % otherwise don't send it gen_tcp:close(Client) %erlang:port_close(Client) end, accept(ListenSock); _Error -> ok end

Any suggestions/help? Thanks!

Because we're using PF_PACKET sockets, we can't call gen_tcp:accept/1. The ethernet frame will be sent to our process as port data. The frame can be parsed using pkt (https://github.com/msantos/pkt).

For sending the frame, use packet:send/3.

Something like this:

accept(ListenSock, IfIndex, Port) ->
    receive
        {Port, {data, Data}} ->
             Frame = pkt:decapsulate(Data),
             Tmp = rule_database:lookup_rule_table(Frame),
             if Tmp ->
                 %if the lookup is positive, send it back
                 % XXX will return {ok, Size} on partial write
                 ok = packet:send(ListenSock, Ifindex, Data);
             true ->
                 % otherwise don't send it

                 % Could generate a RST here
                 %gen_tcp:close(Client)
                 %erlang:port_close(Client)
             end,
             accept(ListenSocket, IfIndex, Port);
        _Error ->
            ok
    end.

If the device is in the middle between the client and server, instead of closing the socket you could generate a RST in both directions to shut down the connection.

If any of that isn't clear or if you have any questions, please let me know!

amuessig commented 8 years ago

Thanks for your helpful hints! Sending RST will be implemented later on. First, there should be a working prototype. I don't know why, but I don't get any data send back, I think they are not even received. This is the code I have figured out for now:

init({Parent, Port}) -> register(?MODULE,self()), proc_lib:init_ack(Parent, {ok, self()}), {ok, ListenSock} = procket:open(Port, [{protocol, 16#0008}, {type, raw}, {family, packet}]), IfIndex = packet:ifindex(ListenSock, "vnf-eth0"), ok = packet:bind(ListenSock, IfIndex), accept(ListenSock, IfIndex, Port).

accept(ListenSock, IfIndex, Port) ->
    io:fwrite("Method accept is called\n"),
    receive
    {Port, {data, Data}} -> 
        Frame = pkt:decapsulate(Data),
        io:fwrite("Frame:", Frame, "\n"),
        Tmp = rule_database:lookup_rule_table(Frame),
        if Tmp ->
            {ok, Size} = packet:send(ListenSock, IfIndex, Data);
        true ->
            erlang:port_close(IfIndex)
        end,
        accept(ListenSock, IfIndex, Port);
    _Error ->
        ok
end.

I am getting the output "Method accept is called", but never "Frame: [...]". Did I misunderstand something? I was sending pings to the vm, which was working. When calling the server by opening a TCP connection to download data, these data are not put back again. I do not even get the message that something was received.

Thanks!

EDIT: Yes, the interface is run in promisc. mode.

msantos commented 8 years ago

Here's a working example:

-module(resend).

-export([init/1]).

init(Dev) ->
    {ok, ListenSock} = procket:open(0, [{protocol, 16#0008}, {type, raw}, {family, packet}]),
    IfIndex = packet:ifindex(ListenSock, Dev),
    ok = packet:bind(ListenSock, IfIndex),
    Port = erlang:open_port({fd, ListenSock, ListenSock}, [binary,stream]),
    accept(ListenSock, IfIndex, Port).

accept(ListenSock, IfIndex, Port) ->
    io:fwrite("Method accept is called\n"),
    receive
        {Port, {data, Data}} ->
            Frame = pkt:decapsulate(Data),
            error_logger:info_report(Frame),
            ok = packet:send(ListenSock, IfIndex, Data),
            accept(ListenSock, IfIndex, Port);
        _Error ->
            ok
    end.

A few things to note:

Hope that clears things up a bit, feel free to post here if you run into any other problems.

amuessig commented 8 years ago

Thanks a lot, works perfectly. When sending a RST, do I have to build my frame manually or is your framework offering this? I only found RST for TCP/UDP sockets.

Appreciating your support! :+1:

msantos commented 8 years ago

There's an old example here: https://gist.github.com/msantos/446057#file-rst-erl And some explanation: http://blog.listincomprehension.com/2010/06/fun-with-raw-sockets-in-erlang-abusing.html

You'll probably need to swap the MAC address of your MITM host as the source MAC address when sending the RSTs.

amuessig commented 8 years ago

It's working very well.

however, I couldn't find anything how to forward frames/packets/segments to the applications of the host where my Erlang is running. In example, when I am using a distributed Erlang, I want to be able to set it up from remote. Currently I only send out the frames or I drop them. But this also drops the packets for starting erlang processes remotely.

msantos commented 8 years ago

It's working very well.

Great!

however, I couldn't find anything how to forward frames/packets/segments to the applications of the host where my Erlang is running.

In example, when I am using a distributed Erlang, I want to be able to set it up from remote. Currently I only send out the frames or I drop them. But this also drops the packets for starting erlang processes remotely.

Probably the simplest way is to match any packets with the source or destination set to the IP address of the MITM host and ignore them. Maybe something like this:

https://github.com/msantos/herp/blob/master/src/herp.erl#L127

amuessig commented 8 years ago

Yeah, it worked perfectly. Do you have any idea how to send ARP requests? In your packet file, ( https://github.com/msantos/procket/blob/master/src/packet.erl ), you show up a way for looking up the mac address in the ARP table of the host, but I cannot find any way to get the MAC address if it is not at the host's 'ARP' cache. Any idea of that or do I have to send ARP packets on my own?

msantos commented 8 years ago

Sure, it is pretty simple. Construct an ARP packet then write it to a PF_PACKET or BPF socket: https://github.com/msantos/farp/blob/master/src/farp.erl#L192

To populate the ARP cache, open any network connection to the host. For example, send a UDP packet to some random port on the host.

If you want more control, you could do an ARP probe or gratuitous ARP using the target host's IP address then sniff the reply from the host.