hard limits on FAR and PDR in the XDM

[neeels]

I am evaluating eUPF for high volume use. I notice that the config options pdr_map_size:1024 far_map_size:1024 seem to have an actual hard limit at 1024. I cannot get more tunnels out of eUPF when I configure all of these to 2048. I still get just 512 tunnels, only a different error message.

With 1024 configured, it is:

2024/03/08 02:03:23 INF Can't put FAR: pool is empty
2024/03/08 02:03:23 INF Rejecting Session Establishment Request from: pool is empty (error in applying IEs)

With 2048, the error also starts to appear from tunnel 512 on, only this time the error is:

2024/03/08 02:10:52 INF Can't put FAR: update: key too big for map: argument list too long
2024/03/08 02:10:52 INF Rejecting Session Establishment Request from: update: key too big for map: argument list too long (error in applying IEs)

So is there a hard limit of 512 tunnels that eUPF can set up? Where does it come from, why is it there, and can we raise this limit somehow? A typical UPF operation can easily extend two orders above this limit, so this could be a show stopper for us.

I would be very glad to hear a developer's insights on this. Many thanks!

[pirog-spb]

Hi Neels @neeels,

Currently, the most reliable way to configure FAR/QER/PDR containers size is to set corresponding constants in the source code. See cmd/ebpf/xdp/pdr.h:

#define PDR_MAP_UPLINK_SIZE 1024
#define PDR_MAP_DOWNLINK_IPV4_SIZE 1024
#define PDR_MAP_DOWNLINK_IPV6_SIZE 1024
#define FAR_MAP_SIZE 1024

But you can also try to set resize_ebpf_maps: true flag in the configuration in order to make qer_map_size, far_map_size, pdr_map_size variables work.

Unfortunately, setting ebpf map size via config doesn't work sometimes. So if changing configuration parameters doesn't seem to be working in your case, you can still modify source code constants, as a fallback alternative.

Best wishes, Alex

[neeels]

Excellent, thanks for this feedback! I was completely unaware of the resize_ebpf_maps option.

I am wondering why the default sizes are so low -- are there known adverse effects that these low numbers aim to avoid?

What do you think are the boundaries of map sizes to choose? Will it be below 10k, or is 100k fine? What about a million?

How will I notice that I picked too large a number? Could it be evil, like a sudden random system crash at an unknown time, or a sublime percentage of failure that could go unnoticed?

Many Thanks!

~N

[pirog-spb]

The default size set so low because lack of load testing, currently.

I suppose, there is no problem technically to increase limits up to 1m and more. It mostly depends on available RAM size.

If you pick too large map size, eUPF won't start due to corresponding error while creating to large ebpf map.

If eUPF becomes up and running with maps size you set, it should work as expected without any side effects.

-- BR, Alex

[neeels]

Cool, many thanks for your feedback!