search

Exa-Networks / exabgp

The BGP swiss army knife of networking
Other
2.09k stars 447 forks source link

JSON generation bug for neighbor-changes #269

Closed pavel-odintsov closed 9 years ago

pavel-odintsov commented 9 years ago

Hello...

I'm so sorry but I have found new bag again.

I have this error when I run flow announce from another ExaBGP instance:

env exabgp.log.level=DEBUG  exabgp.daemon.user=root exabgp.tcp.bind="0.0.0.0" exabgp.tcp.port=179 exabgp.daemon.daemonize=false exabgp.daemon.pid=/var/run/exabgp.pid exabgp.log.destination=/var/log/exabgp.log sbin/exabgp /etc/exabgp_blackhole.conf
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | reactor       | Performing reload of exabgp 3.4.10
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading | group Core_v4 {
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading |     hold-time 180;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading |     local-as 1234;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading |     peer-as 65001;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading |     router-id 10.0.3.115;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading |     graceful-restart 1200;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading | 
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading |     neighbor 10.0.3.114 {
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading |         local-address 10.0.3.115;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading |         description "Quagga";
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading | 
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading |         process stdout {
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading |             neighbor-changes;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading | 
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading |             receive {
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading |                 parsed;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading |                 update;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading |             }
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading | 
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading |             encoder json;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading |             run /usr/src/fastnetmon/src/scripts/exabgp_queue_writer.py ;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading |         }
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading |     }
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | loading | }
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | parsing | configuration | 'group' 'core_v4' '{'
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | parsing | group         | 'hold-time' '180' ';'
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | parsing | group         | 'local-as' '1234' ';'
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | parsing | group         | 'peer-as' '65001' ';'
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | parsing | group         | 'router-id' '10.0.3.115' ';'
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | parsing | group         | 'graceful-restart' '1200' ';'
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | parsing | group         | 'neighbor' '10.0.3.114' '{'
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | parsing | neighbor      | 'local-address' '10.0.3.115' ';'
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | parsing | neighbor      | 'description' '"quagga"' ';'
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | parsing | neighbor      | 'process' 'stdout' '{'
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | parsing | process       | 'neighbor-changes' ';'
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | parsing | process       | 'receive' '{'
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | parsing | receive       | 'parsed' ';'
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | parsing | receive       | 'update' ';'
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | parsing | receive       | '}'
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | parsing | process       | 'encoder' 'json' ';'
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | parsing | process       | 'run' '/usr/src/fastnetmon/src/scripts/exabgp_queue_writer.py ' ';'
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | parsing | process       | '}'
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | parsing | neighbor      | '}'
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | neighbor 10.0.3.114 {
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |   description "quagga";
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |   router-id 10.0.3.115;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |   host-name filter;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |   domain-name fvee;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |   local-address 10.0.3.115;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |   local-as 1234;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |   peer-as 65001;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |   hold-time 180;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |   group-updates: True;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |   auto-flush: true;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |   adj-rib-out: true;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |   ttl-security: ;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | 
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |   capability {
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     asn4 enable;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     route-refresh disable;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     graceful-restart 1200;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     add-path disable;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     multi-session disable;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     operational disable;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     aigp disable;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |   }
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |   family {
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     inet4 unicast;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     inet4 multicast;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     inet4 nlri-mpls;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     inet4 mpls-vpn;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     inet4 rtc;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     inet4 flow;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     inet4 flow-vpn;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     inet6 unicast;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     inet6 multicast;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     inet6 nlri-mpls;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     inet6 mpls-vpn;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     inet6 flow;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     inet6 flow-vpn;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     l2vpn vpls;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     l2vpn evpn;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |   }
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |   process {
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     receive {
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |       parsed;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |       update;
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |     }
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration |   }
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | }
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | 
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | 
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | configuration | parsing | group         | '}'
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | reactor       | New peer setup: neighbor 10.0.3.114 local-ip 10.0.3.115 local-as 1234 peer-as 65001 router-id 10.0.3.115 family-allowed in-open
Sun, 17 May 2015 23:40:19 | WARNING  | 8513   | configuration | Loaded new configuration successfully
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | reactor       | Listening for BGP session(s) on 0.0.0.0:179
Sun, 17 May 2015 23:40:19 | INFO     | 8513   | processes     | Forked process stdout
{ "exabgp": "3.5.0", "time": 1431898824, "host" : "filter.fv.ee", "pid" : 8513, "ppid" : 8435, "counter": 1, "type": "state", "neighbor": { "address": { "local": "10.0.3.115", "peer": "10.0.3.114" }, "asn": { "local": "1234", "peer": "65001" }, "state": "down", "reason": "connection to peer failed" } }
{   u'counter': 1,
    u'exabgp': u'3.5.0',
    u'host': u'filter.fv.ee',
    u'neighbor': {   u'address': {   u'local': u'10.0.3.115',
                                     u'peer': u'10.0.3.114'},
                     u'asn': {   u'local': u'1234', u'peer': u'65001'},
                     u'reason': u'connection to peer failed',
                     u'state': u'down'},
    u'pid': 8513,
    u'ppid': 8435,
    u'time': 1431898824,
    u'type': u'state'}
{ "exabgp": "3.5.0", "time": 1431898830, "host" : "filter.fv.ee", "pid" : 8513, "ppid" : 8435, "counter": 2, "type": "state", "neighbor": { "address": { "local": "10.0.3.115", "peer": "10.0.3.114" }, "asn": { "local": "1234", "peer": "65001" }, "state": "down", "reason": "connection to peer failed" } }
{   u'counter': 2,
    u'exabgp': u'3.5.0',
    u'host': u'filter.fv.ee',
    u'neighbor': {   u'address': {   u'local': u'10.0.3.115',
                                     u'peer': u'10.0.3.114'},
                     u'asn': {   u'local': u'1234', u'peer': u'65001'},
                     u'reason': u'connection to peer failed',
                     u'state': u'down'},
    u'pid': 8513,
    u'ppid': 8435,
    u'time': 1431898830,
    u'type': u'state'}
{ "exabgp": "3.5.0", "time": 1431898831, "host" : "filter.fv.ee", "pid" : 8513, "ppid" : 8435, "counter": 3, "type": "state", "neighbor": { "address": { "local": "10.0.3.115", "peer": "10.0.3.114" }, "asn": { "local": "1234", "peer": "65001" }, "state": "connected" } }
{   u'counter': 3,
    u'exabgp': u'3.5.0',
    u'host': u'filter.fv.ee',
    u'neighbor': {   u'address': {   u'local': u'10.0.3.115',
                                     u'peer': u'10.0.3.114'},
                     u'asn': {   u'local': u'1234', u'peer': u'65001'},
                     u'state': u'connected'},
    u'pid': 8513,
    u'ppid': 8435,
    u'time': 1431898831,
    u'type': u'state'}
{ "exabgp": "3.5.0", "time": 1431898833, "host" : "filter.fv.ee", "pid" : 8513, "ppid" : 8435, "counter": 4, "type": "state", "neighbor": { "address": { "local": "10.0.3.115", "peer": "10.0.3.114" }, "asn": { "local": "1234", "peer": "65001" }, "state": "down", "reason": "connection to peer failed" } }
{   u'counter': 4,
    u'exabgp': u'3.5.0',
    u'host': u'filter.fv.ee',
    u'neighbor': {   u'address': {   u'local': u'10.0.3.115',
                                     u'peer': u'10.0.3.114'},
                     u'asn': {   u'local': u'1234', u'peer': u'65001'},
                     u'reason': u'connection to peer failed',
                     u'state': u'down'},
    u'pid': 8513,
    u'ppid': 8435,
    u'time': 1431898833,
    u'type': u'state'}
{ "exabgp": "3.5.0", "time": 1431898833, "host" : "filter.fv.ee", "pid" : 8513, "ppid" : 8435, "counter": 5, "type": "state", "neighbor": { "address": { "local": "10.0.3.115", "peer": "10.0.3.114" }, "asn": { "local": "1234", "peer": "65001" }, "state": "up" } }
{   u'counter': 5,
    u'exabgp': u'3.5.0',
    u'host': u'filter.fv.ee',
    u'neighbor': {   u'address': {   u'local': u'10.0.3.115',
                                     u'peer': u'10.0.3.114'},
                     u'asn': {   u'local': u'1234', u'peer': u'65001'},
                     u'state': u'up'},
    u'pid': 8513,
    u'ppid': 8435,
    u'time': 1431898833,
    u'type': u'state'}
{ "exabgp": "3.5.0", "time": 1431898833, "host" : "filter.fv.ee", "pid" : 8513, "ppid" : 8435, "counter": 6, "type": "update", "neighbor": { "address": { "local": "10.0.3.115", "peer": "10.0.3.114" }, "asn": { "local": "1234", "peer": "65001" }, "direction": "receive", "message": { "update": { "attribute": { "origin": "igp", "as-path": [ 65001 ], "confederation-path": [], "extended-community": [ 0x800900000000000A ] }, "announce": { "ipv4 flow": { "no-nexthop": { "flow-0": { "destination-ipv4": [ "10.0.0.2/32" ], "source-ipv4": [ "10.0.0.1/32" ], "destination-port": [ "=3128" ], "source-port": [ "=1234" ], "tcp-flags": [ "=syn" ], "fragment": [ "=is-fragment" ], "string": "flow destination-ipv4 10.0.0.2/32 source-ipv4 10.0.0.1/32 destination-port =3128 source-port =1234 tcp-flags =syn fragment =is-fragment" } } } } } } } }
Traceback (most recent call last):
  File "/usr/src/fastnetmon/src/scripts/exabgp_queue_writer.py", line 48, in <module>
    decoded_update = json.load(io) 
  File "/usr/lib/python2.7/json/__init__.py", line 290, in load
    **kw)
  File "/usr/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
    obj, end = self.scan_once(s, idx)
ValueError: Expecting , delimiter: line 1 column 401 (char 400)

Here you can find my script: https://gist.github.com/pavel-odintsov/c620639766720c4610f0

ExaBGP config:

group Core_v4 {
    hold-time 180;
    local-as 1234;
    peer-as 65001;
    router-id 10.0.3.115;
    graceful-restart 1200;

    neighbor 10.0.3.114 {
        local-address 10.0.3.115;
        description "Quagga";

        process stdout {
            neighbor-changes;

            receive {
                parsed;
                update;
            }   

            encoder json;
            run /usr/src/fastnetmon/src/scripts/exabgp_queue_writer.py ;
        }   
    }   
}
pavel-odintsov commented 9 years ago

And I found place where broken json (value without quotes) located:

"extended-community": [ 0x800900000000000A ]

I received your commits about extended-community and can't find any bugs.

thomas-mangin commented 9 years ago

Damn, JSON does not let you express number as hexadecimal, this is why it was sent as integer before ! I broke it .. need to revert the patch !

pavel-odintsov commented 9 years ago

What about custom to string converter?

thomas-mangin commented 9 years ago

Sorry - a community is a number and unlike normal one, there is no good syntax to express them. hex would have give a fixed size number - but base 10 it is.