search

vmxdev / damper

Linux userspace traffic shaper
ISC License
38 stars 4 forks source link

HTB shaping support #1

Open mmanoj opened 7 years ago

mmanoj commented 7 years ago

Hi,

Thanks for the nice initiative, I would like to know any plan to support HTB shaping. Also advice is this support 10G rates.

Thank you, Manoj M

vmxdev commented 7 years ago

I would like to know any plan to support HTB shaping

There is mechanism for setting priorities "by hand" - see bymark.c. If shaper gets packet with mark (marks can be set by iptables -j MARK --set-mark), it sets priority to this packet as specified in damper.conf, "bymark <mark> <priority>".

If packet has not mark, or mark is not listed in damper.conf, priority in this module will be set to minimal.

There is no hierarchy and percents in module, only plain priorities. Don't know, do we need the complication here?

is this support 10G rates

I don't have such a hardware, so can't say anything. Looks like it's not a good idea to shape traffic in userspace on such a high rates. Damper was written and tested in SOHO environment only.

However, user-space IDS/IPS such as Suricata works at 10G and higher (with some tweaks). So I guess it's possible to shape traffic in userspace on a high rates, but some work is required. And it may be not a NFQUEUE solution, raw sockets looks more promising for high network speeds

mmanoj commented 7 years ago

Hi, Thanks for detail clarifications.Really appreciated.My plan is to make high speed shaper as it's required by industry. Do you have any guide line where to start such requirement, So I can start looking and we can put open source project for the same.

What do you think? I found interesting frameworks possibility can use.

https://github.com/scylladb/seastar

https://software.intel.com/en-us/blogs/2015/06/12/user-space-networking-fuels-nfv-performance

vmxdev commented 7 years ago

I guess, choosing framework depends on which type of shaper you want to get.

There is few approaches in traffic shaping - traditional manual bandwith control by TCP/IP header fields, shaping by application/protocol type ( using some DPI such as https://github.com/ntop/nDPI ) or completely automatic shaping like https://en.wikipedia.org/wiki/CoDel

For shapers like CoDel (or like damper's flow entropy calculations) there is no need to use full TCP/IP stack frameworks (with it's overhead).

However, overhead may not be as significant. I think, the best way to test is to actually write some code.

DPDK has some QoS code, there is a nice article about it: http://dpdk.org/doc/guides/prog_guide/qos_framework.html

mmanoj commented 7 years ago

Hi, Thanks again for the guidelines, I'm looking to have DPI enable QoS with per IP based (individual) policy qos. I also think DPDK is one of the candidate.However it's need specific HW and custom code. On the other side I saw XDP evolving with eBPF.

What do you think about XDP+eBPF approach for my requirement. I'm new to this subject and still learning.

vmxdev commented 7 years ago

I'm looking to have DPI enable QoS with per IP based (individual) policy qos

It may be not so simple, but why not try?

What do you think about XDP+eBPF approach for my requirement.

Don't know about it.

Ok, after some googling I decided to write some code using raw sockets (PACKET_MMAP for capturing described here: https://www.kernel.org/doc/Documentation/networking/packet_mmap.txt). It should work on any Linux with reasonable performance.

Unfortunatelly, I do not have much time, so can't promise that I will do anything at all. If there will be any results I will write here

mmanoj commented 7 years ago

Hi,

Thanks for the effort and positive response. I'm looking the XDP+eBPF path and update you any good news. Other wise will extend the project better way more community get benefited. If no one start such project yet.

Keep in touch.