search

frenetic-lang / pyretic

The Pyretic language and runtime system
http://frenetic-lang.org/pyretic/
159 stars 99 forks source link

pox/pyretic bug on tcp traffic e.g., http/iperf #7

Closed joshreich closed 11 years ago

joshreich commented 11 years ago

details sent in email and discussed over chat today - will fill in more later.

monsanto commented 11 years ago

For me the HTTP test works fine (on mininet 2.0).

h1: --2013-01-18 20:54:18--  http://10.0.0.2/
h1: Connecting to 10.0.0.2:80... connected.
h1: HTTP request sent, awaiting response... 200 OK
h1: Length: unspecified
h1: Saving to: `STDOUT'
h1: 
h1:      0K                                                        0.00 =0s
h1: 
h1: 2013-01-18 20:54:18 (0.00 B/s) - written to stdout [0]
h1:

It doesn't print out a directory listing like it will if you uncomment the call('controller ptcp: &', shell=True) line, but the HTTP server still works. The directory listing seems to be an artifact of that call command; if I start mininet manually with the reference controller and start the python webserver and do a wget, I also don't get a directory listing. I bet the shell=True keyword argument changes the working directory or something weird.

monsanto commented 11 years ago

I don't see iperf in the repo, just http_test.

joshreich commented 11 years ago

It's not in the repo - I was doing the tests manually. Here's my setup for iperf. screen 1: pox.py --no-cli pyretic/examples/hub.py screen 2: ./pyretic/mininet.sh --switch ovsk --topo=linear,3 --test iperf

What I happens on my VM. screen 1: INFO:packet:(tcp parse) warning TCP data offset too long or too short 15 (repeated many times) screen 2: stalls at *\ Iperf: testing TCP bandwidth between h1 and h3

On Jan 19, 2013, at 12:31 AM, Christopher Monsanto notifications@github.com wrote:

I don't see iperf in the repo, just http_test.

— Reply to this email directly or view it on GitHub.

joshreich commented 11 years ago

hmm, very strange. the http server itself should work fine, but the http_test script should give the exact same results whether you use the stock controller (by uncommenting) or the pyretic controller. If I understand you correctly, you are getting different output. If so, why are you saying things work fine?

either way, please try the iperf setup I just added to this issue and let me know what happens on your system.

thanks!

On Jan 18, 2013, at 11:59 PM, Christopher Monsanto notifications@github.com wrote:

For me the HTTP test works fine (on mininet 2.0). It doesn't actually

h1: --2013-01-18 20:54:18-- http://10.0.0.2/ h1: Connecting to 10.0.0.2:80... connected. h1: HTTP request sent, awaiting response... 200 OK h1: Length: unspecified h1: Saving to: `STDOUT' h1: h1: 0K 0.00 =0s h1: h1: 2013-01-18 20:54:18 (0.00 B/s) - written to stdout [0] h1: It doesn't print out a directory listing like it will if you uncomment the call('controller ptcp: &', shell=True) line, but the HTTP server still works. The directory listing seems to be an artifact of that call command; if I start mininet manually with the reference controller and start the python webserver and do a wget, I also don't get a directory listing.

— Reply to this email directly or view it on GitHub.

monsanto commented 11 years ago

What I am saying is that if I run sudo mn (no flag for remote controller) and h1 python -m SimpleHTTPServer 80 and h2 wget h1 I also get no output. The only time I get output is if I do it in the Python script by uncommenting the line. Either way, you can verifiably see that the server is communicating back using the HTTP protocol. The only difference is that when you run the controller via that mininet python script, it responds with a different document. As I said, I think the reason is the shell=True keyword, which may be doing something to the permissions/working directory which causes a directory listing to be presented.

monsanto commented 11 years ago

OK, I can reproduce that iperf problem. I'll see what I can do.

monsanto commented 11 years ago

The iperf seems to be a POX bug. If you look at get_packet_payload, immediately returning packet["payload"] works fine, but returning p_begin.pack() (before we mutate p), it doesn't work. That means that POX's own unpack and pack commands aren't inverses of each other. I'll see what I can do in the next 2 hours and tomorrow morning.

Edit: here is an example. Note that the hub doesn't modify any of its packets.

ipdb> payload
'\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x01\x08\x00E\x00\x00r\x0f\xd5@\x00@\x06\x16\xae\n\x00\x00\x01\n\x00\x00\x03\xbc\xa7\x13\x89\xd0\n-i[}\xe2\xc4\x80\x10\x00\x1d,;\x00\x00\x01\x01\x08\n\x00\x93\xb5\xc0\x00\x91"\xe923456789012345678901234567890123456789012345678901234567890123'
ipdb> packet["payload"]
'\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x01\x08\x00E\x00\x05\x9e\x0f\xd5@\x00@\x06\x11\x82\n\x00\x00\x01\n\x00\x00\x03\xbc\xa7\x13\x89\xd0\n-i[}\xe2\xc4\x80\x10\x00\x1d#x\x00\x00\x01\x01\x08\n\x00\x93\xb5\xc0\x00\x91"\xe923456789012345678901234567890123456789012345678901234567890123'
monsanto commented 11 years ago

Also it takes forever to run the iperf test without the error. This is no doubt a symptom of not using microflows yet.

monsanto commented 11 years ago

Check this:

ipdb> packet.next.iplen
64
ipdb> packet.pack()
'\x00\x00\x00\x00\x00\x03\x00\x00\x00\x00\x00\x01\x08\x00E\x10\x00>@{@\x00@\x06\xe6+\n\x00\x00\x01\n\x00\x00\x03\xbe\xe9\x13\x89\x95N\xab\xe9\x17\x08\x14\xa8\xb0\x10\x00\x1d\xc02\x00\x00\x01\x01\x08\n\x00\x99\xec\xbe\x00\x99\xec\xb3\x01\x01\x17\x08\x14\xa7\x17\x08\x14\xa8'
ipdb> packet.next.iplen
62

why on earth would serializing the packet change the iplen???

monsanto commented 11 years ago

So it apparently mutates the iplen to the "true length". But the tcp frame is given a different length when serialized. Looks like some of the tcp options it isn't parsing correctly, as I have one option serializing to \x01\x05\n and the other serializing to just \x01. (Which accounts for the two bytes missing)

joshreich commented 11 years ago

uch, sounds like more pox spaghetti mess

On Jan 20, 2013, at 2:18 AM, Christopher Monsanto notifications@github.com wrote:

So it apparently mutates the iplen to the "true length". But the tcp frame is given a different length when serialized. Looks like some of the tcp options it isn't parsing correctly, as I have one option serializing to \x01\x05\n and the other serializing to just \x01.

— Reply to this email directly or view it on GitHub.

joshreich commented 11 years ago

ah, so I think I see what happened and I don't think it is the shell=True thing (although that might affect something as well). you are running the command

h2 wget h1 whereas the script is running the command h2 wget -O - h1

try running the later command and see if that's the case.

On Jan 19, 2013, at 11:21 PM, Christopher Monsanto notifications@github.com wrote:

What I am saying is that if I run sudo mn (no flag for remote controller) and h1 python -m SimpleHTTPServer 80 and h2 wget h1 I also get no output. The only time I get output is if I do it in the Python script by uncommenting the line. Either way, you can verifiably see that the server is communicating back using the HTTP protocol. The only difference is that when you run the controller via that mininet python script, it responds with a different document. As I said, I think the reason is the shell=True keyword, which may be doing something to the permissions/working directory which causes a directory listing to be presented.

— Reply to this email directly or view it on GitHub.

monsanto commented 11 years ago

ffs, what an annoying bug. When POX finds a SACK TCP option, it parses it in correctly, but when it serializes it, it forgets to prefix the option data with the fact that its an SACK! Hence, we lose two bytes, and the resulting serialized packet makes no sense, and our original packet is destroyed as well by POX mutating the fields of the packet. Fix incoming...

This fix doesn't "fix" POX's lame mutation behavior, but at least POX should serialize the packet correctly, and thus the mutations will be harmless.

monsanto commented 11 years ago

Oh, and it seems like POX also doesn't calculate checksums correctly when serializing either

monsanto commented 11 years ago

OK, I simply can't figure out this checksum stuff. I've run the algorithm manually and get the same results as POX. I don't know what's going on. Anyway, you can pull the latest code from my POX repository to hopefully fix your iperf problem.

monsanto commented 11 years ago

Josh I can no longer reproduce your HTTP test bug. Even when I run http_test with the controller ptcp: line uncommented (of course checking for existing python servers or controllers), I get a successful HTTP transaction with no output. I again ran sudo mn (which starts the ref controller) and performed this experiment manually, getting

mininet> h1 python -m SimpleHTTPServer 80 &
mininet> h2 wget -O - h1
--2013-01-21 01:54:47--  http://10.0.0.1/
Connecting to 10.0.0.1:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 0 [text/html]
Saving to: `STDOUT'

     0K                                                        0.00 =0s

2013-01-21 01:54:47 (0.00 B/s) - written to stdout [0/0]

just like I got with the pyretic hub. So, no matter what I can't seem to get the directory listing again.

I'm convinced the pyretic stack works fine: sudo apt-get install apache2, service apache2 stop, and then run mininet manually pyretic/mininet.sh --topo=linear,3 and pox.py --no-cli pyretic/examples/hub.py

now h1 service apache2 start and h3 wget -O - h1. This should work, but it will be comically slow. (3.93 B/s here). We absolutely need microflows to do any sort of measurement.

joshreich commented 11 years ago

So this is fairly strange, but I think of minor importance at the moment. Let's schedule a meeting for Wed. and we can figure this out then.

On Jan 21, 2013, at 5:03 AM, Christopher Monsanto notifications@github.com wrote:

Josh I can no longer reproduce your HTTP test bug. Even when I run http_test with the controller ptcp: line uncommented (of course checking for existing python servers or controllers), I get a successful HTTP transaction with no output. I again ran sudo mn (which starts the ref controller) and performed this experiment manually, getting

mininet> h1 python -m SimpleHTTPServer 80 & mininet> h2 wget -O - h1 --2013-01-21 01:54:47-- http://10.0.0.1/ Connecting to 10.0.0.1:80... connected. HTTP request sent, awaiting response... 200 OK Length: 0 [text/html] Saving to: `STDOUT'

 0K                                                        0.00 =0s

2013-01-21 01:54:47 (0.00 B/s) - written to stdout [0/0] just like I got with the pyretic hub. So, no matter what I can't seem to get the directory listing again.

I'm convinced the pyretic stack works fine: sudo apt-get install apache2, service apache2 stop, and then run mininet manually pyretic/mininet.sh --topo=linear,3 and pox.py --no-cli pyretic/examples/hub.py

now h1 service apache2 start and h3 wget -O - h1. This should work, but it will be comically slow. (3.93 B/s here). We absolutely need microflows to do any sort of measurement.

— Reply to this email directly or view it on GitHub.