search

luigirizzo / netmap

Automatically exported from code.google.com/p/netmap
BSD 2-Clause "Simplified" License
1.84k stars 534 forks source link

Assistance with Tcpreplay bad addr/len #582

Open fklassen opened 5 years ago

fklassen commented 5 years ago

I'm having the following issue when I attempt to replay packets > 1500 bytes on 9000 MTU interface (Linux) with tcpreplay/netmap.

Jan 21 19:53:02 replay11 kernel: [  159.743146] 382.663361 [ 183] ixgbe_netmap_txsync       bad addr/len ring 0 slot 193 idx 195 len 3736
Jan 21 19:53:02 replay11 kernel: [  159.799148] 382.719341 [ 183] ixgbe_netmap_txsync       bad addr/len ring 0 slot 195 idx 197 len 2742
Jan 21 19:53:02 replay11 kernel: [  159.854932] 382.775099 [ 183] ixgbe_netmap_txsync       bad addr/len ring 0 slot 199 idx 201 len 2742
Jan 21 19:53:02 replay11 kernel: [  159.910695] 382.830837 [ 183] ixgbe_netmap_txsync       bad addr/len ring 0 slot 201 idx 203 len 5515
Jan 21 19:53:02 replay11 kernel: [  159.966462] 382.886577 [ 183] ixgbe_netmap_txsync       bad addr/len ring 0 slot 203 idx 205 len 2666
Jan 21 19:53:02 replay11 kernel: [  160.022216] 382.942307 [ 183] ixgbe_netmap_txsync       bad addr/len ring 0 slot 284 idx 286 len 2964
Jan 21 19:53:03 replay11 kernel: [  160.791758] 383.711497 [ 183] ixgbe_netmap_txsync       bad addr/len ring 0 slot 314 idx 316 len 3539
Jan 21 19:53:03 replay11 kernel: [  160.848241] 383.767955 [ 183] ixgbe_netmap_txsync       bad addr/len ring 0 slot 316 idx 318 len 3472

As far as I can tell, this is coming from this debug macro defined as 1.

Is this correct? Or is there something I have to do to TX on interfaces with MTU > 1500.

The overall effect is that performance is very poor, and logs are filling up.

vmaffione commented 5 years ago

You need to increase /sys/module/netmap/parameters/buf_size.

giuseppelettieri commented 5 years ago

Yes, the defauit buffer size is 2048. You can either increase that (no change needed in the app, but possibly wasteful) or support NS_MOREFRAG in tcpreplay. Basically, you need to use several consecutive slots for a single message and set NS_MOREFRAG in the slot flags of all but the last slot.

fklassen commented 5 years ago

I am having Tcpreplay crashes when I increase the buffer size dynamically. Does the value have to be a power of two? I am attempting to accept 9018 frames. I'm not sure if I have to alter Tcpreplay source code to make this work.

fklassen commented 5 years ago

or support NS_MOREFRAG in tcpreplay. Basically, you need to use several consecutive slots for a single message and set NS_MOREFRAG in the slot flags of all b

I'll add a ticket to add that flag. Question: Does the flag work dynamicallay, e.g. if a 1500 byte packet comes in, will it only take one 2048 byte buffer?

giuseppelettieri commented 5 years ago

The buffer size can be any value between 64 and 65536. it doesn't need to be a power of 2, but the number you enter may rounded to a multiple of a cacheline (you should see a message in the logs). The new value is not used while there are netmap applications still running.

giuseppelettieri commented 5 years ago

Does the flag work dynamicallay, e.g. if a 1500 byte packet comes in, will it only take one 2048 byte buffer?

it is a per-slot flag. If you have a packet that fits a single slot, you simply don't set the flag. If you have a packet that needs 4 slots, you set the flag in the first three.

fklassen commented 5 years ago

it is a per-slot flag. If you have a packet that fits a single slot, you simply don't set the flag. If you have a packet that needs 4 slots, you set the flag in the first three.

That makes sense. Thanks for clarifying. Opened appneta/tcpreplay:#534 to track this issue.