search

aregm / nff-go

NFF-Go -Network Function Framework for GO (former YANFF)
BSD 3-Clause "New" or "Revised" License
1.38k stars 156 forks source link

Error with two flows, number of cores isn't enough. #698

Closed sitilge closed 4 years ago

sitilge commented 4 years ago

I was trying to set up two flows, both duplex on separate ports (0 and 1). The VM has been assigned 6 cores.

f1, err := flow.SetReceiver(0)
flow.CheckFatal(err)
flow.CheckFatal(flow.SetHandler(f1, handler1, nil))
flow.CheckFatal(flow.SetSender(f1, 0)

f2, err := flow.SetReceiver(1)
flow.CheckFatal(err)
flow.CheckFatal(flow.SetHandler(f2, handler2, nil))
flow.CheckFatal(flow.SetSender(f2, 1)

However, what I am getting is

DEBUG: Start new instance for senderPortThread1
WARNING: Can't start new clone for senderPortThread1 instance 0
ERROR: failed with message and code: Requested number of cores isn't enough. (18)
scheduler start failed

I assume that the system wants more cores, right? Or what can be the problem then? It is stated in the docs: https://github.com/intel-go/nff-go/wiki/Building-Packet-Processing-Graph

Also note, that each flow function use at least one CPU core from the set of cores that was given to SystemInit.

It is not said how many exactly though. So I the snippet above is using at least 6 cores just for flow functions. And as regards affinity - is each function bound to the some specific cores and runs non-preemptively (by other FFs)? If so, it means that no core is left for the scheduler, right? For example, when running only the first flow, the number of cores used is 4 = 3 FFs + 1 scheduler, which makes sense.

Update: having 6 FFs works with 7 cores allocated.

aregm commented 4 years ago

@sitilge In your case each receiver is 1 core, each sender is 1 core, 1 FF and 1 scheduler resulting total 7 cores needed.