noxrepo / nox

The NOX Controller
Other
139 stars 85 forks source link

boost::asio::streambuf too long #5

Closed Tyrrr closed 11 years ago

Tyrrr commented 12 years ago

I want to test the performance of nox with different modules, so I'm using Cbench(http://www.openflow.org/wk/index.php/Oflops) in throughput mode but after a few seconds of running Cbench I get the following error messages: 146439|event-dispatcher|ERR:Event ofp_packet_in processing leaked an exception: boost::asio::streambuf too long 146440|event-dispatcher|ERR:Extra information: Throw location unknown (consider using BOOST_THROW_EXCEPTION) Dynamic exception type:boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<std::length_error> > std::exception::what: boost::asio::streambuf too long The source code of the module: http://pastebin.com/851D8R2e I don't get these errors with the switch module, so I first thought that my module was generating packets too fast but actually this module generates less packets than the switch module. Any idea what's happening here?

rafaelob commented 11 years ago

I got same error! Is there any solution for this??? i am using export LD_PRELOAD=/usr/lib/libtcmalloc_minimal.so.4.1.0

I am running an comparable model, and because this i got poor results with nox

ghost commented 11 years ago

I have the same errors. My configuration is a virtual machine with reserved hardware: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz with turbo mode 3.80Ghz, 2 core and 4 threads. Ram 8Gb The operating system is Ubuntu 12.04.2 LTS x86_64 with kernel version 2.6.32-11-pve #1 SMP Wed Apr 11 07:17:05 CEST 2012 x86_64 GNU/Linux. Boost Library: v1.48.0.2 (libboost-all-dev)

I am launching local or remote test of cbench in throughput with this command:

cbench -s 100 -l 110 -w 10 -i 1 -c (server ip) -t

Nox is started:

./nox_core -i ptcp:6633 switch -t 4

After 5-10 loops I have a lot of errors like this:

237483|event-dispatcher|ERR:Event ofp_packet_in processing leaked an exception: boost::asio::streambuf too long
237484|event-dispatcher|ERR:Extra information:
Throw in function (unknown)
Dynamic exception type: boost::exception_detail::clone_impl<boost::exception_detail::error_info_injector<std::length_error> >
std::exception::what: boost::asio::streambuf too long

Cbench after the error returns 0 flows/seconds.

ghost commented 11 years ago

Do you need other information?

tootoonchian commented 11 years ago

Asd0r, what if you run cbench locally? Are the two boxes directly connected?

ghost commented 11 years ago

The two server are connected through a gigabit switch. The two virtual machine are in different server. I have both server running another test that will last all the night (here is 11pm). I will try tomorrow to start local and remote throughput test for nox because I do not rember exactly.

tootoonchian commented 11 years ago

What this error means is that in your setting the bottleneck is flushing the write buffer. Either the benchmarker is not reading the responses fast enough or NOX is not passing the bits down to the TCP buffer fast enough. It manifests in form of an exception because the send() call doesn't expect the write buffer to be full. I'll change it to return with a failure if the write buffer doesn't have enough space to fully write the message.

tootoonchian commented 11 years ago

Could you try the following patch and let me know if it fixes the issue?

https://gist.github.com/tootoonchian/5352392

ghost commented 11 years ago

I have tired two test quickly and seems to work well. I will start a very long test now.

rafaelob commented 11 years ago

When i tried apply the patch, i got this:

git apply txbuf_len_exception.patch txbuf_len_exception.patch:47: trailing whitespace.

fatal: corrupt patch at line 107

2013/4/10 Asd0r notifications@github.com

I have tired two test quickly and seems to work well. I will start a very long test now.

— Reply to this email directly or view it on GitHubhttps://github.com/noxrepo/nox/issues/5#issuecomment-16169458 .

ghost commented 11 years ago

Erase or add a new line at the end of the file.

ghost commented 11 years ago

The new patch is working

tootoonchian commented 11 years ago

Thanks! Unless someone objects I'll commit the patch later today.

ghost commented 11 years ago

Wait until tomorrow so I could test in latency mode to verify if is working correctly with the other cbench modality.

tootoonchian commented 11 years ago

Got confirmations from several users, so I pushed the patch.