IPLookup fails to add more than 32706 routes

[trozet]

When using IPLookup module, no matter how large the max_rules are set to, I cannot add more than 32706 routes:

localhost:10514 $ run /root/bess/bessctl/conf/trozet/l3fwd BESS_FLOWS=16400 WARNING: The current pipeline will be reset. Are you sure? (type "yes") yes Using 16400 flows... (envvar "BESS_FLOWS") prefix length set to 32 starting addresses set to: 12.0.0.3, 11.0.0.3 Added 32706 rou Error: Unhandled exception in the configuration script (most recent call last)
File "/root/bess/bessctl/conf/trozet/l3fwd.bess", line 53, in fib.add(prefix=str(east_addr), prefix_len=prefix_len, gate=11) File "/root/trozet_bess/bessctl/../pybess/module.py", line 37, in self.bess.run_module_command(self.name, cmd, arg_type, kwargs) File "/root/trozet_bess/bessctl/../pybess/bess.py", line 493, in run_module_command response = self._request('ModuleCommand', request) File "/root/trozet_bess/bessctl/../pybess/bess.py", line 272, in _request raise self.Error(code, errmsg, query=name, query_arg=req_dict) Error: rpm_lpm_add() failed BESS daemon response - errno=28 (ENOSPC: No space left on device) command: add command_arg: {'gate': 11, 'prefix': '11.0.64.0', 'prefix_len': 32} module: ip_lookup0 query: ModuleCommand query_arg: {'cmd': 'add', 'name': 'ip_lookup0', 'arg': {'type_url': 'type.googleapis.com/bess.pb.IPLookupCommandAddArg', 'value': '\n\t11.0.64.0\x10 \x18\x0b'}}

Note when using BESS_FLOWS=16400, the IPLookup module will be initialized with BESS_FLOWS*2+1= 32801, but at 32706 add fails. Full script:

https://github.com/trozet/bess/commit/9f8d3bf177e333c38f995770555f22f1a88fca44

[sangjinhan]

The IPLookup module is simply a wrapper of DPDK LPM library. Because of the way it is implemented, there is a limitation on how many prefixes that are smaller than /24 can be installed. The limitation can be tuned with max_tbl8s. https://doc.dpdk.org/guides/prog_guide/lpm_lib.html has some documentation.

[trozet]

@sangjinhan thanks for the link. Makes sense that after /24 table is used the tbl8s is used for last 8 bits. I changed the IPLookup to be fib = IPLookup(max_rules=flows2+1, max_tbl8s=flows2) and was able to add 100k routes. However, I think this is another documentation issue. The wiki says: "Maximum number of IP prefixes with smaller than /24 "

However, this really is the maximum number of IP prefixes longer than /24. Do you agree?