L3VPN are not getting stored in DB

[ayalaalex]

I recently updated the docker image to use lastest (Release 0.14.0.0) and now the l3vpn prefixes are not getting stored in the DB. Perhaps the addition of large communities broke something?

[TimEvens]

@ayalaalex , the migration script in the AIO and mysql containers only migrated from the last schema version "1.22" to "1.23". It's likely that's the issue.

Can you double check the docker container log files /var/log/mysql-consumer*.log to see which error is being reported? I suspect it's a DB error regarding the missing large_community_list column.

Can you apply the following to your DB to correct the schema:

alter table path_attrs add column large_community_list varchar(3000) DEFAULT NULL;

drop view IF EXISTS v_routes;
CREATE  VIEW v_routes AS
       SELECT  if (length(rtr.name) > 0, rtr.name, rtr.ip_address) AS RouterName,
                if(length(p.name) > 0, p.name, p.peer_addr) AS PeerName,
                r.prefix AS Prefix,r.prefix_len AS PrefixLen,
                path.origin AS Origin,r.origin_as AS Origin_AS,path.med AS MED,
                path.local_pref AS LocalPref,path.next_hop AS NH,path.as_path AS AS_Path,
                path.as_path_count AS ASPath_Count,path.community_list AS Communities,
                path.ext_community_list AS ExtCommunities,path.large_community_list AS LargeCommunities,
                path.cluster_list AS ClusterList,
                path.aggregator AS Aggregator,p.peer_addr AS PeerAddress, p.peer_as AS PeerASN,r.isIPv4 as isIPv4,
                p.isIPv4 as isPeerIPv4, p.isL3VPNpeer as isPeerVPN,
                r.timestamp AS LastModified, r.first_added_timestamp as FirstAddedTimestamp,r.prefix_bin as prefix_bin,
                r.path_id, r.labels,
                r.hash_id as rib_hash_id,
                r.path_attr_hash_id as path_hash_id, r.peer_hash_id, rtr.hash_id as router_hash_id,r.isWithdrawn,
                r.prefix_bits,r.isPrePolicy,r.isAdjRibIn
        FROM bgp_peers p JOIN rib r ON (r.peer_hash_id = p.hash_id)
            JOIN path_attrs path ON (path.hash_id = r.path_attr_hash_id and path.peer_hash_id = r.peer_hash_id)
            JOIN routers rtr ON (p.router_hash_id = rtr.hash_id)
       WHERE r.isWithdrawn = False;

drop view IF EXISTS v_all_routes;
CREATE  VIEW v_all_routes AS
       SELECT  if (length(rtr.name) > 0, rtr.name, rtr.ip_address) AS RouterName,
                if(length(p.name) > 0, p.name, p.peer_addr) AS PeerName,
                r.prefix AS Prefix,r.prefix_len AS PrefixLen,
                path.origin AS Origin,r.origin_as AS Origin_AS,path.med AS MED,
                path.local_pref AS LocalPref,path.next_hop AS NH,path.as_path AS AS_Path,
                path.as_path_count AS ASPath_Count,path.community_list AS Communities,
                path.ext_community_list AS ExtCommunities,path.large_community_list AS LargeCommunities,
                path.cluster_list AS ClusterList,
                path.aggregator AS Aggregator,p.peer_addr AS PeerAddress, p.peer_as AS PeerASN,r.isIPv4 as isIPv4,
                p.isIPv4 as isPeerIPv4, p.isL3VPNpeer as isPeerVPN,
                r.timestamp AS LastModified,r.first_added_timestamp as FirstAddedTimestamp,r.prefix_bin as prefix_bin,
                r.path_id, r.labels,
                r.hash_id as rib_hash_id,
                r.path_attr_hash_id as path_hash_id, r.peer_hash_id, rtr.hash_id as router_hash_id,r.isWithdrawn,
                r.prefix_bits
        FROM bgp_peers p JOIN rib r ON (r.peer_hash_id = p.hash_id)
            JOIN path_attrs path ON (path.hash_id = r.path_attr_hash_id and path.peer_hash_id = r.peer_hash_id)
            JOIN routers rtr ON (p.router_hash_id = rtr.hash_id);

drop view IF EXISTS v_routes_history;
CREATE VIEW v_routes_history AS
  SELECT
                rtr.name as RouterName, rtr.ip_address as RouterAddress,
            p.name AS PeerName,
                pathlog.prefix AS Prefix,pathlog.prefix_len AS PrefixLen,
                path.origin AS Origin,path.origin_as AS Origin_AS,
                    path.med AS MED,path.local_pref AS LocalPref,path.next_hop AS NH,
                path.as_path AS AS_Path,path.as_path_count AS ASPath_Count,path.community_list AS Communities,
                path.ext_community_list AS ExtCommunities,path.large_community_list AS LargeCommunities,
                path.cluster_list AS ClusterList,path.aggregator AS Aggregator,p.peer_addr AS PeerAddress,
                p.peer_as AS PeerASN,  p.isIPv4 as isPeerIPv4, p.isL3VPNpeer as isPeerVPN,
                pathlog.id,pathlog.timestamp AS LastModified,
               pathlog.path_attr_hash_id as path_attr_hash_id, pathlog.peer_hash_id, rtr.hash_id as router_hash_id
        FROM path_attr_log pathlog
                 STRAIGHT_JOIN path_attrs path
                                 ON (pathlog.path_attr_hash_id = path.hash_id AND
                                         pathlog.peer_hash_id = path.peer_hash_id)
                 STRAIGHT_JOIN bgp_peers p ON (pathlog.peer_hash_id = p.hash_id)
             STRAIGHT_JOIN routers rtr ON (p.router_hash_id = rtr.hash_id)
                  ORDER BY id Desc;

drop view IF EXISTS v_routes_withdraws;
CREATE VIEW v_routes_withdraws AS
SELECT  rtr.name as RouterName, rtr.ip_address as RouterAddress,
    p.name AS PeerName,
        log.prefix AS Prefix,log.prefix_len AS PrefixLen,
        path.origin AS Origin,path.origin_as AS Origin_AS,path.med AS MED,path.local_pref AS LocalPref,
        path.next_hop AS NH,path.as_path AS AS_Path,path.as_path_count AS ASPath_Count,
        path.community_list AS Communities,
        path.ext_community_list AS ExtCommunities,path.large_community_list AS LargeCommunities,
        path.cluster_list AS ClusterList,
        path.aggregator AS Aggregator,p.peer_addr AS PeerAddress,p.peer_as AS PeerASN,
        p.isIPv4 AS isPeerIPv4,p.isL3VPNpeer AS isPeerVPN,log.id AS id,log.timestamp AS LastModified,
        log.path_attr_hash_id AS path_attr_hash_id,log.peer_hash_id AS peer_hash_id,rtr.hash_id AS router_hash_id
    FROM withdrawn_log log
         STRAIGHT_JOIN path_attrs path ON (path.hash_id = log.path_attr_hash_id and path.peer_hash_id = log.peer_hash_id)
         STRAIGHT_JOIN bgp_peers p ON (log.peer_hash_id = p.hash_id)
         LEFT JOIN routers rtr ON (p.router_hash_id = rtr.hash_id)
    ORDER BY log.timestamp desc;

drop view IF EXISTS v_l3vpn_routes;
CREATE VIEW v_l3vpn_routes AS
    select if((length(rtr.name) > 0),rtr.name,rtr.ip_address) AS RouterName,
    if((length(p.name) > 0),p.name,p.peer_addr) AS PeerName,
    r.rd AS RD,r.prefix AS Prefix,r.prefix_len AS PrefixLen,path.origin AS Origin,
    r.origin_as AS Origin_AS,path.med AS MED,path.local_pref AS LocalPref,
    path.next_hop AS NH,path.as_path AS AS_Path,
    path.as_path_count AS ASPath_Count,path.community_list AS Communities,
    path.ext_community_list AS ExtCommunities,path.large_community_list AS LargeCommunities,
  path.cluster_list AS ClusterList,
    path.aggregator AS Aggregator,p.peer_addr AS PeerAddress,p.peer_as AS PeerASN,
    r.isIPv4 AS isIPv4,p.isIPv4 AS isPeerIPv4,p.isL3VPNpeer AS isPeerVPN,
    r.timestamp AS LastModified,r.first_added_timestamp AS FirstAddedTimestamp,
    r.prefix_bin AS prefix_bin,r.path_id AS path_id,r.labels AS labels,r.hash_id AS rib_hash_id,
    r.path_attr_hash_id AS path_hash_id,r.peer_hash_id AS peer_hash_id,
    rtr.hash_id AS router_hash_id,r.isWithdrawn AS isWithdrawn,
    r.prefix_bits AS prefix_bits,r.isPrePolicy AS isPrePolicy,r.isAdjRibIn AS isAdjRibIn
     from bgp_peers p
               join l3vpn_rib r on (r.peer_hash_id = p.hash_id)
        join path_attrs path on (path.hash_id = r.path_attr_hash_id and path.peer_hash_id = r.peer_hash_id)
              join routers rtr on (p.router_hash_id = rtr.hash_id)
      where  r.isWithdrawn = 0;

[ayalaalex]

@TimEvens I don't see any errors in the mysql-consumer logs, and the mysql schema already has the large_community field. I see in the logs:

2018-03-21T02:02:43.964216 | DEBUG | parseBGP.cpp[00548] | UpdateDBL3Vpn | 10.20.1.4: adding vpn=29.1.2.0 len=24 2018-03-21T02:02:43.964228 | DEBUG | MsgBusImpl_kafka.cpp[00404] | produce | rtr=100.1.1.3: Producing message: topic=openbmp.parsed.l3vpn key=733df964f693f7de426e2ff97a2f00aa, msg size = 286 2018-03-21T02:02:43.964235 | DEBUG | MsgBusImpl_kafka.cpp[01583] | send_bmp_raw | rtr=100.1.1.3: Producing bmp raw message: topic=openbmp.bmp_raw key=c2d7d27e666fc9ccc0188245c9fa2646, msg size = 145

No Issues there, but I check in the mysql: MariaDB [openBMP]> select * from l3vpn_rib; Empty set (0.00 sec)

mysql-consumer log: 2018-03-21 02:12:31 [main] INFO org.openbmp.MySQLConsumerApp - -- STATS -- thread: 0 read: 351 queue: 0
2018-03-21 02:12:31 [main] INFO org.openbmp.MySQLConsumerApp - collector messages: 0 2018-03-21 02:12:31 [main] INFO org.openbmp.MySQLConsumerApp - router messages: 0 2018-03-21 02:12:31 [main] INFO org.openbmp.MySQLConsumerApp - peer messages: 11 2018-03-21 02:12:31 [main] INFO org.openbmp.MySQLConsumerApp - reports messages: 0 2018-03-21 02:12:31 [main] INFO org.openbmp.MySQLConsumerApp - base attribute messages: 170 2018-03-21 02:12:31 [main] INFO org.openbmp.MySQLConsumerApp - unicast prefix messages: 0 2018-03-21 02:12:31 [main] INFO org.openbmp.MySQLConsumerApp - l3vpn prefix messages: 170 2018-03-21 02:12:31 [main] INFO org.openbmp.MySQLConsumerApp - LS node messages: 0 2018-03-21 02:12:31 [main] INFO org.openbmp.MySQLConsumerApp - LS link messages: 0 2018-03-21 02:12:31 [main] INFO org.openbmp.MySQLConsumerApp - LS prefix messages: 0

[TimEvens]

are you using the aio container?

can you also do a docker exec ls -l /usr/local/ | grep build to get the buld number you are using?

[ayalaalex]

Yes I'm using the aio container:

docker exec openbmp ls -l /usr/local/ | grep build -rw-r--r-- 1 root root 0 Mar 6 20:16 build-106

When I use this build it works: docker exec openbmp ls -l /usr/local/ | grep build -rw-r--r-- 1 root root 0 Feb 8 18:27 build-103

[TimEvens]

@ayalaalex , ok, that build is bad. There was an update to that build on 3/13 (build 107). I believe if you upgrade it should start working.

[ayalaalex]

Thanks Tim! that worked.