osrg / gobgp

BGP implemented in the Go Programming Language
https://osrg.github.io/gobgp/
Apache License 2.0
3.67k stars 699 forks source link

Very high memory consumption (full BGP table) #2331

Open dylancmds opened 4 years ago

dylancmds commented 4 years ago

I'm using alice-lg with a goBGP backend which collects a full BGP table from a route-reflector in our network. This configuration tries to consume over 50Gb of ram on the system! Is there anything I can do to improve this?

gobgpd.conf
 -------------
[global.config]
  as = 1234
  router-id = "1.2.3.4"

[[neighbors]]
[neighbors.config]
  neighbor-address = "1.2.3.5"
  peer-as = 1234

[[neighbors]]
[neighbors.config]
  neighbor-address = "1:2:3::6"
  peer-as = 1234

[[rpki-servers]]
  [rpki-servers.config]
    address = "1.2.3.7"
    port = 1234

 alice.conf
 ---------------
[server]
 listen_http = :80
 enable_prefix_lookup = true
 enable_neighbors_status_refresh = true
 asn = 1234

[housekeeping]
 interval = 5
 force_release_memory = true

[pagination]
 routes_filtered_page_size = 250
 routes_accepted_page_size = 250
 routes_not_exported_page_size = 250

[rpki]
 enabled = true

[neighbours_columns]
 address = Neighbour
 asn = ASN
 state = State
 Uptime = Uptime
 Description = Description
 routes_received = Routes Received
 routes_filtered = Filtered

[routes_columns]
 network = Network
 gateway = Gateway
 interface = Interface
 metric = Metric
 bgp.as_path = AS Path

[lookup_columns]
 network = Network
 gateway = Gateway
 neighbour.asn = ASN
 neighbour.description = Description
 bgp.as_path = AS Path
 routeserver.name = RS

[source.debug0]
 name = debug0
 group = AMS

[source.debug0.gobgp]
 host = 127.0.0.1:50051
 processing_timeout = 300

When running gobgpd on its own, with peers established, I only consume under 2Gb of ram. For reference, this is an ubuntu docker container named 'alice-lg'

root@3a3fe0a973c7:/$ gobgp neighbor
Peer              AS  Up/Down State       |#Received  Accepted
1.2.3.5         1234 00:05:58 Establ      |   819821    819821
1:2:3::6        1234 00:06:03 Establ      |    95955     95955

CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT    MEM %               NET I/O             BLOCK I/O           PIDS
3a3fe0a973c7        alice-lg            0.32%               1.24GiB / 62.86GiB   1.97%               384MB / 21.3MB      1.52GB / 251MB      39

When starting alice-lg, it skyrockets, I had to kill it here as it kept climbing:


CONTAINER ID        NAME                CPU %               MEM USAGE / LIMIT     MEM %               NET I/O             BLOCK I/O           PIDS
3a3fe0a973c7        alice-lg            100.46%             43.25GiB / 62.86GiB   68.80%              384MB / 21.3MB      1.52GB / 251MB      71

top - 11:53:24 up 6 days,  2:26,  0 users,  load average: 3.44, 4.00, 2.71
Tasks:  14 total,   1 running,  12 sleeping,   1 stopped,   0 zombie
%Cpu(s):  7.7 us,  1.0 sy,  0.0 ni, 91.3 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
MiB Mem :  64371.3 total,   1129.1 free,  59302.6 used,   3939.5 buff/cache
MiB Swap:   8192.0 total,   6626.9 free,   1565.1 used.  10923.4 avail Mem

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND
 1501 root      20   0   20.4g  18.3g  10320 S  99.7  29.1   3:08.22 alice-lg
 1471 root      20   0   25.5g  24.9g  12172 S   0.0  39.6  10:14.24 gobgpd

As seen here, alice-lg and gobgpd are consuming similar amounts of high memory. Should this be happening? Is something configured incorrectly? I understand a full BGP table will consume a decent amount of mem, but I have a feeling this is a bit excessive. Please let me know if there's logs or outputs I can collect to help with this issue.

I have issues opened in both alice-lg/alice-lg and osrg/gobgp as I'm not sure which piece is the culprit. Forgive me, I'm a net-eng by trade and only stand up unix applications occasionally, so if there's something I'm doing wrong feel free to let me know.

serejkus commented 3 years ago

related issue: https://github.com/alice-lg/alice-lg/issues/44