search

Exa-Networks / exabgp

The BGP swiss army knife of networking
Other
2.09k stars 448 forks source link

api processes TypeError: unhashable type: 'Neighbor' on self._dispatch[message_id] #944

Closed jkldgoefgkljefogeg closed 4 years ago

jkldgoefgkljefogeg commented 4 years ago

neighbor api send packets or neighbor api send consolidate causes exabgp to crash loop

https://github.com/Exa-Networks/exabgp/blob/d006a34a/lib/exabgp/reactor/api/processes.py#L338

10:21:36 | 5176   | reactor       | ExaBGP version : 4.1.2-d006a34a
10:21:36 | 5176   | reactor       | Python version : 3.8.0 (default, Oct 28 2019, 16:14:01)  [GCC 8.3.0]
10:21:36 | 5176   | reactor       | System Uname   : #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019
10:21:36 | 5176   | reactor       | System MaxInt  : 9223372036854775807
10:21:36 | 5176   | reactor       |         
10:21:36 | 5176   | reactor       |         
10:21:36 | 5176   | reactor       |         
10:21:36 | 5176   | reactor       |         
10:21:36 | 5176   | reactor       | <class 'TypeError'>
10:21:36 | 5176   | reactor       | unhashable type: 'Neighbor'
10:21:36 | 5176   | reactor       | Traceback (most recent call last):
10:21:36 | 5176   | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/peer.py", line 545, in _run
10:21:36 | 5176   | reactor       |     for action in self._establish():
10:21:36 | 5176   | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/peer.py", line 343, in _establish
10:21:36 | 5176   | reactor       |     for sent_open in self._send_open():
10:21:36 | 5176   | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/peer.py", line 299, in _send_open
10:21:36 | 5176   | reactor       |     for message in self.proto.new_open():
10:21:36 | 5176   | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/protocol.py", line 318, in new_open
10:21:36 | 5176   | reactor       |     for _ in self.write(sent_open):
10:21:36 | 5176   | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/protocol.py", line 165, in write
10:21:36 | 5176   | reactor       |     self._to_api('send',message,raw)
10:21:36 | 5176   | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/protocol.py", line 150, in _to_api
10:21:36 | 5176   | reactor       |     self.peer.reactor.processes.message(self.peer.neighbor,direction,message,negotiated,raw[:19],raw[19:])
10:21:36 | 5176   | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/api/processes.py", line 292, in closure
10:21:36 | 5176   | reactor       |     return function(self,*args)
10:21:36 | 5176   | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/api/processes.py", line 338, in message
10:21:36 | 5176   | reactor       |     self._dispatch[message_id](self,neighbor,direction,message,negotiated,header,*body)
10:21:36 | 5176   | reactor       | TypeError: unhashable type: 'Neighbor'
jkldgoefgkljefogeg commented 4 years ago

I didn't send anything over named pipe. exabgp is crashing looping (doesn't not bind on tcp/179) with api send packet/consolidate enabled (only 1 IPv4 BGP with v4 unicast AFI configured)

        api {
            processes [exabgp_pipe];
            neighbor-changes;
            negotiated;
            fsm;
            signal;
            receive {
                parsed;
                packets;
                consolidate;
                open;
                update;
                notification;
                keepalive;
                refresh;
                operational;
            }
            send {
                parsed;
                packets;
                consolidate;
                open;
                update;
                notification;
                keepalive;
                refresh;
                operational;
            }
        }
thomas-mangin commented 4 years ago

Initial test does not replicate - will look at your config.

process my-process {
    run ./run/syslog-1.py;
    encoder json;
}

template {
    neighbor test {
        api {
            processes [ my-process ];
            neighbor-changes;
            receive {
                parsed;
                update;
            }
            send {
                consolidate;
                packets;
            }
        }
    }
}
neighbor 127.0.0.1 {
    description "will pass received routes to the program";
    router-id 10.0.0.2;
    local-address 127.0.0.1;
    local-as 65533;
    peer-as 65533;
    hold-time 180;

    static {
    }
}
env \
     exabgp_debug_rotate=true \
     exabgp_debug_configuration=true \
     exabgp_tcp_bind='' \
     exabgp_tcp_port=1790 \
     INTERPRETER=/Applications/Xcode.app/Contents/Developer/usr/bin/python3  \
         ./sbin/exabgp -d -p \
             ./etc/exabgp/xxxx

env exabgp_tcp_port=1790 /Applications/Xcode.app/Contents/Developer/usr/bin/python3 ./qa/sbin/bgp --echo```
thomas-mangin commented 4 years ago

also working with

process my-process {
    run ./run/syslog-1.py;
    encoder json;
}

template {
    neighbor test {
        api {
            processes [ my-process ];
            neighbor-changes;
            negotiated;
            fsm;
            signal;
            receive {
                parsed;
                packets;
                consolidate;
                open;
                update;
                notification;
                keepalive;
                refresh;
                operational;
            }
            send {
                parsed;
                packets;
                consolidate;
                open;
                update;
                notification;
                keepalive;
                refresh;
                operational;
            }
        }
    }
}
neighbor 127.0.0.1 {
    description "will pass received routes to the program";
    router-id 10.0.0.2;
    local-address 127.0.0.1;
    local-as 65533;
    peer-as 65533;
    hold-time 180;

    static {
    }
}
thomas-mangin commented 4 years ago

So please provide me your full configuration and full output as I can not reproduce - if you can not / do not I can not help (as I can not reproduce !). You can emaill me at first @ last .com if you prefer.

thomas-mangin commented 4 years ago

Tried with python3.8 on Mac (I was testing with 3.7) can still not reproduce.

jkldgoefgkljefogeg commented 4 years ago 

process exabgp_pipe {
    run /usr/bin/python3.8 /home/user/tmp/exabgp/exabgp.py;
    encoder json;
}

template {
    neighbor example-v4 {
        router-id 192.0.2.0;
        local-address 192.0.2.0;
        local-as 30216;

        family {
            ipv4 unicast;
        }

        api {
            processes [exabgp_pipe];
            neighbor-changes;
            negotiated;
            fsm;
            signal;
            receive {
                parsed;
                packets;
                consolidate;
                open;
                update;
                notification;
                keepalive;
                refresh;
                operational;
            }
            send {
                parsed;
                packets;
                consolidate;
                open;
                update;
                notification;
                keepalive;
                refresh;
                operational;
            }
        }

    }
}

neighbor 192.0.2.7 {
    inherit example-v4;
    description vultr-v4;
    peer-as 64515;
    # passive;
    # hold-time <seconds>;
    md5-password *********;
    # ttl-security;
    # group-updates;
    # aigp <truthvalue>;
    # route-refresh <truthvalue>;
    # graceful-restart <...>;
    # multi-session <truthvalue);
    # add-path <path>;
    # auto-flush <truthvalue>;
    # adj-rib-out <truthvalue>;
}
04:37:26 | 8435   | reactor       | loaded new configuration successfully
04:37:26 | 8435   | process       | forked process exabgp_pipe
04:37:26 | 8435   | process       | forked process api-internal-cli-114c9da4
04:37:26 | 8435   | reactor       | initialising connection to peer-1
04:37:26 | 8435   | outgoing-1    | attempting connection to 169.254.169.254:179
04:37:26 | 8435   | reactor       |         
04:37:26 | 8435   | reactor       | ********************************************************************************
04:37:26 | 8435   | reactor       | EXABGP MISBEHAVED / HELP US FIX IT
04:37:26 | 8435   | reactor       | ********************************************************************************
04:37:26 | 8435   | reactor       |         
04:37:26 | 8435   | reactor       | Sorry, you encountered a problem with ExaBGP, as the problem only affects one
04:37:26 | 8435   | reactor       | peer, we are trying to keep the program running.
04:37:26 | 8435   | reactor       |         
04:37:26 | 8435   | reactor       | There are a few things you can do to help us (and yourself):
04:37:26 | 8435   | reactor       | - make sure you are running the latest version of the code available at
04:37:26 | 8435   | reactor       |   https://github.com/Exa-Networks/exabgp/releases/latest
04:37:26 | 8435   | reactor       | - if so report the issue on https://github.com/Exa-Networks/exabgp/issues
04:37:26 | 8435   | reactor       |   so it can be fixed (github can be searched for similar reports)
04:37:26 | 8435   | reactor       |         
04:37:26 | 8435   | reactor       | PLEASE, when reporting, do include as much information as you can:
04:37:26 | 8435   | reactor       | - do not obfuscate any data (feel free to send us a private  email with the
04:37:26 | 8435   | reactor       |   extra information if your business policy is strict on information sharing)
04:37:26 | 8435   | reactor       |   https://github.com/Exa-Networks/exabgp/wiki/FAQ
04:37:26 | 8435   | reactor       | - if you can reproduce the issue, run ExaBGP with the command line option -d
04:37:26 | 8435   | reactor       |   it provides us with much needed information to fix problems quickly
04:37:26 | 8435   | reactor       | - include the information presented below
04:37:26 | 8435   | reactor       |         
04:37:26 | 8435   | reactor       | Should you not receive an acknowledgment of your issue on github (assignement,
04:37:26 | 8435   | reactor       | comment, or similar) within a few hours, feel free to email us to make sure
04:37:26 | 8435   | reactor       | it was not overlooked. (please keep in mind the authors are based in GMT/Europe)
04:37:26 | 8435   | reactor       |         
04:37:26 | 8435   | reactor       | ********************************************************************************
04:37:26 | 8435   | reactor       | -- Please provide ALL the information below on :
04:37:26 | 8435   | reactor       | -- https://github.com/Exa-Networks/exabgp/issues
04:37:26 | 8435   | reactor       | ********************************************************************************
04:37:26 | 8435   | reactor       |         
04:37:26 | 8435   | reactor       | ExaBGP version : 4.1.2-d006a34a
04:37:26 | 8435   | reactor       | Python version : 3.8.0 (default, Oct 28 2019, 16:14:01)  [GCC 8.3.0]
04:37:26 | 8435   | reactor       | System Uname   : #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019
04:37:26 | 8435   | reactor       | System MaxInt  : 9223372036854775807
04:37:26 | 8435   | reactor       |         
04:37:26 | 8435   | reactor       |         
04:37:26 | 8435   | reactor       |         
04:37:26 | 8435   | reactor       |         
04:37:26 | 8435   | reactor       | <class 'TypeError'>
04:37:26 | 8435   | reactor       | unhashable type: 'Neighbor'
04:37:26 | 8435   | reactor       | Traceback (most recent call last):
04:37:26 | 8435   | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/peer.py", line 545, in _run
04:37:26 | 8435   | reactor       |     for action in self._establish():
04:37:26 | 8435   | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/peer.py", line 343, in _establish
04:37:26 | 8435   | reactor       |     for sent_open in self._send_open():
04:37:26 | 8435   | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/peer.py", line 299, in _send_open
04:37:26 | 8435   | reactor       |     for message in self.proto.new_open():
04:37:26 | 8435   | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/protocol.py", line 318, in new_open
04:37:26 | 8435   | reactor       |     for _ in self.write(sent_open):
04:37:26 | 8435   | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/protocol.py", line 165, in write
04:37:26 | 8435   | reactor       |     self._to_api('send',message,raw)
04:37:26 | 8435   | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/protocol.py", line 150, in _to_api
04:37:26 | 8435   | reactor       |     self.peer.reactor.processes.message(self.peer.neighbor,direction,message,negotiated,raw[:19],raw[19:])
04:37:26 | 8435   | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/api/processes.py", line 292, in closure
04:37:26 | 8435   | reactor       |     return function(self,*args)
04:37:26 | 8435   | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/api/processes.py", line 338, in message
04:37:26 | 8435   | reactor       |     self._dispatch[message_id](self,neighbor,direction,message,negotiated,header,*body)
04:37:26 | 8435   | reactor       | TypeError: unhashable type: 'Neighbor'
04:37:26 | 8435   | reactor       |         
04:37:26 | 8435   | reactor       | ********************************************************************************
04:37:26 | 8435   | reactor       | -- Please provide _ALL_ the information above on :
04:37:26 | 8435   | reactor       | -- https://github.com/Exa-Networks/exabgp/issues
04:37:26 | 8435   | reactor       | ********************************************************************************
04:37:26 | 8435   | reactor       |         
04:37:26 | 8435   | outgoing-1    | peer reset, message [] error[]
thomas-mangin commented 4 years ago

@jkldgoefgkljefogeg thank you

Can you also provide the output of your /home/user/tmp/exabgp/exabgp.py program as it is what is causing this error and I need to see which command is badly parsed.

thomas-mangin commented 4 years ago

@jkldgoefgkljefogeg are you still able to help ?

jkldgoefgkljefogeg commented 4 years ago 
cat tmp/exabgp/exabgp.py 
import sys

with open('/home/user/tmp/exabgp/everyone/exabgp_api.txt', 'a') as f:
    while True:
            line = sys.stdin.readline()
            f.write(line)
            f.flush()
thomas-mangin commented 4 years ago

@jkldgoefgkljefogeg two things: the script should not end. Without the text file /home/user/tmp/exabgp/everyone/exabgp_api.txt I am not any wiser

jkldgoefgkljefogeg commented 4 years ago

The script does not end (confirmed once I comment out send packets or send consolidate)

There is nothing in /home/user/tmp/exabgp/everyone/exabgp_api.txt. The script just appends to this file.

It doesn't seem like the crash is related to the process script. I get the same error with the following script if neighbor api send packets or neighbor api send consolidate is enabled in exabgp config

import sys

import requests

while True:
    line = sys.stdin.readline()
    if len(line) < 1:
        continue
    r = requests.post('http://webhook.site/#!/552a5465-211f-4c56-a22c-a3c6b6ea5646', data=line)
thomas-mangin commented 4 years ago

thank you. Sorry I answered too quickly. I need to know what is received by exabgp and what packet causes the issue. Running with -d will display this information.

jkldgoefgkljefogeg commented 4 years ago

tcpdump on port 179

previous output was with -d. The crash happens before BGP going into established state, thus not much log

tcpdump: listening on ens3, link-type EN10MB (Ethernet), capture size 262144 bytes                                                                                                  [90/833]
08:57:28.346898 IP (tos 0xc0, ttl 2, id 29240, offset 0, flags [DF], proto TCP (6), length 72)
    169.254.169.254.49091 > bgp.example.com.bgp: Flags [S], cksum 0x2556 (incorrect -> 0xf92f), seq 1087010161, win 29200, options [nop,nop,md5 shared secret not supplied with -M, can't chec
k - 6c1335785efc3ee5c93e58c2cf2ad33e,mss 1460,nop,nop,sackOK,nop,wscale 11], length 0                                                                                                      
08:57:29.361117 IP (tos 0xc0, ttl 2, id 29241, offset 0, flags [DF], proto TCP (6), length 72)
    169.254.169.254.49091 > bgp.example.com.bgp: Flags [S], cksum 0x2556 (incorrect -> 0xf92f), seq 1087010161, win 29200, options [nop,nop,md5 shared secret not supplied with -M, can't chec
k - 6c1335785efc3ee5c93e58c2cf2ad33e,mss 1460,nop,nop,sackOK,nop,wscale 11], length 0                                                                                                       
08:57:31.409091 IP (tos 0xc0, ttl 2, id 29242, offset 0, flags [DF], proto TCP (6), length 72)
    169.254.169.254.49091 > bgp.example.com.bgp: Flags [S], cksum 0x2556 (incorrect -> 0xf92f), seq 1087010161, win 29200, options [nop,nop,md5 shared secret not supplied with -M, can't chec
k - 6c1335785efc3ee5c93e58c2cf2ad33e,mss 1460,nop,nop,sackOK,nop,wscale 11], length 0                                                                                                       
08:57:35.441187 IP (tos 0xc0, ttl 2, id 29243, offset 0, flags [DF], proto TCP (6), length 72)       
    169.254.169.254.49091 > bgp.example.com.bgp: Flags [S], cksum 0x2556 (incorrect -> 0xf92f), seq 1087010161, win 29200, options [nop,nop,md5 shared secret not supplied with -M, can't chec
k - 6c1335785efc3ee5c93e58c2cf2ad33e,mss 1460,nop,nop,sackOK,nop,wscale 11], length 0                                                                                                      
08:57:39.283423 IP (tos 0x0, ttl 64, id 18891, offset 0, flags [DF], proto TCP (6), length 72)
    bgp.example.com.40303 > 169.254.169.254.bgp: Flags [S], cksum 0x2556 (incorrect -> 0x021b), seq 4227379343, win 29200, options [nop,nop,md5 shared secret not supplied with -M, can't chec
k - d3989c5cf193954eaccc5537ae0f800d,mss 1460,nop,nop,sackOK,nop,wscale 6], length 0                                                                                                        
08:57:39.283512 IP (tos 0xc0, ttl 255, id 0, offset 0, flags [DF], proto TCP (6), length 72)
    169.254.169.254.bgp > bgp.example.com.40303: Flags [S.], cksum 0x2556 (incorrect -> 0x16c9), seq 222224517, ack 4227379344, win 29200, options [nop,nop,md5 shared secret not supplied wit
h -M, can't check - 32522df9815625aab19426dd512df385,mss 1460,nop,nop,sackOK,nop,wscale 11], length 0                                                                                       
08:57:39.283547 IP (tos 0x0, ttl 64, id 18892, offset 0, flags [DF], proto TCP (6), length 60)
    bgp.example.com.40303 > 169.254.169.254.bgp: Flags [.], cksum 0x254a (incorrect -> 0x3811), seq 1, ack 1, win 457, options [nop,nop,md5 shared secret not supplied with -M, can't check -
19bddefdf9948de416db2ffe8e205f18], length 0                                                                                                                                                
08:57:39.283983 IP (tos 0xc0, ttl 2, id 53892, offset 0, flags [DF], proto TCP (6), length 149)
    169.254.169.254.bgp > bgp.example.com.40303: Flags [P.], cksum 0x25a3 (incorrect -> 0x7d88), seq 1:90, ack 1, win 15, options [nop,nop,md5 shared secret not supplied with -M, can't check
 - 2aa0c106f7fce9645f6c7401d5c85530], length 89: BGP                                                                                                                                        
        Open Message (1), length: 89
          Version 4, my AS 64515, Holdtime 180s, ID 104.156.240.93.example.com                                                                                                                
          Optional parameters, length: 60                                                                                                                                                   
            Option Capabilities Advertisement (2), length: 6
              Multiprotocol Extensions (1), length: 4                                         
                AFI IPv4 (1), SAFI Unicast (1)                                                                                                                                              
                0x0000:  0001 0001                                                  
            Option Capabilities Advertisement (2), length: 2                                
              Route Refresh (Cisco) (128), length: 0                                                                                                                                        
            Option Capabilities Advertisement (2), length: 2                                          
              Route Refresh (2), length: 0                                                    
            Option Capabilities Advertisement (2), length: 6                                                                                                                               
              32-Bit AS Number (65), length: 4
                 4 Byte AS 64515
                0x0000:  0000 fc03
            Option Capabilities Advertisement (2), length: 6
              Multiple Paths (69), length: 4
                AFI IPv4 (1), SAFI Unicast (1), Send/Receive: Receive
                0x0000:  0001 0101
            Option Capabilities Advertisement (2), length: 20
              Unknown (73), length: 18
                no decoder for Capability 73
                0x0000:  106d 6961 3432 382e 7675 6c74 722e 636f
                0x0010:  6d00
            Option Capabilities Advertisement (2), length: 4
              Graceful Restart (64), length: 2
                Restart Flags: [none], Restart Time 3s
                0x0000:  0003
08:57:39.283999 IP (tos 0x0, ttl 64, id 18893, offset 0, flags [DF], proto TCP (6), length 60)                                                                                      [37/833]
    bgp.example.com.40303 > 169.254.169.254.bgp: Flags [.], cksum 0x254a (incorrect -> 0x443a), seq 1, ack 90, win 457, options [nop,nop,md5 shared secret not supplied with -M, can't check -
 9b2f7c7730621036c547c69978fc4aa7], length 0                                                  
08:57:39.307498 IP (tos 0x0, ttl 64, id 18894, offset 0, flags [DF], proto TCP (6), length 60)                                                                                              
    bgp.example.com.40303 > 169.254.169.254.bgp: Flags [R.], cksum 0x254a (incorrect -> 0x6eba), seq 1, ack 90, win 457, options [nop,nop,md5 shared secret not supplied with -M, can't check
- 53722a0e8fb41d44e4fcc05eb062fd08], length 0                                                 
08:57:39.329474 IP (tos 0x0, ttl 64, id 37352, offset 0, flags [DF], proto TCP (6), length 72)                                                                                              
    bgp.example.com.44289 > 169.254.169.254.bgp: Flags [S], cksum 0x2556 (incorrect -> 0x26e6), seq 1441163448, win 29200, options [nop,nop,md5 shared secret not supplied with -M, can't che$
k - 2ce9ce08681d2826a0dfdf5a785e5cb7,mss 1460,nop,nop,sackOK,nop,wscale 6], length 0          
08:57:39.329578 IP (tos 0xc0, ttl 255, id 0, offset 0, flags [DF], proto TCP (6), length 72)                                                                                                
    169.254.169.254.bgp > bgp.example.com.44289: Flags [S.], cksum 0x2556 (incorrect -> 0xd091), seq 295531614, ack 1441163449, win 29200, options [nop,nop,md5 shared secret not supplied wi$
h -M, can't check - 5497d70adb42635f37b624a5cf8d1a9b,mss 1460,nop,nop,sackOK,nop,wscale 11], length 0
08:57:39.329600 IP (tos 0x0, ttl 64, id 37353, offset 0, flags [DF], proto TCP (6), length 60)                                                                                              
    bgp.example.com.44289 > 169.254.169.254.bgp: Flags [.], cksum 0x254a (incorrect -> 0x5257), seq 1, ack 1, win 457, options [nop,nop,md5 shared secret not supplied with -M, can't check -
c875a7488f3da7a101f3eb91e59b6662], length 0                                                   
08:57:39.353209 IP (tos 0x0, ttl 64, id 37354, offset 0, flags [DF], proto TCP (6), length 60)                                                                                              
    bgp.example.com.44289 > 169.254.169.254.bgp: Flags [F.], cksum 0x254a (incorrect -> 0xdbba), seq 1, ack 1, win 457, options [nop,nop,md5 shared secret not supplied with -M, can't check $
 a0ece6eda5fe5d9ca0af9230babfdda6], length 0                                                
08:57:39.354027 IP (tos 0xc0, ttl 255, id 18031, offset 0, flags [DF], proto TCP (6), length 60)                                                                                            
    169.254.169.254.bgp > bgp.example.com.44289: Flags [.], cksum 0x254a (incorrect -> 0xe2b6), seq 1, ack 2, win 15, options [nop,nop,md5 shared secret not supplied with -M, can't check - $
7ebc9a19d86f361821e0af3f7ac6a46], length 0                                                    
08:57:39.577742 IP (tos 0xc0, ttl 2, id 18032, offset 0, flags [DF], proto TCP (6), length 60)                                                                                             
    169.254.169.254.bgp > bgp.example.com.44289: Flags [F.], cksum 0x254a (incorrect -> 0xef07), seq 1, ack 2, win 15, options [nop,nop,md5 shared secret not supplied with -M, can't check -
baeaa3fbfbb1771ebb5f5d411f653b6b], length 0                                                    
08:57:39.577800 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)                                                                                                  
    bgp.example.com.44289 > 169.254.169.254.bgp: Flags [.], cksum 0x9a71 (correct), seq 2, ack 2, win 457, options [nop,nop,md5 shared secret not supplied with -M, can't check - c04f6896a62$
7770e99096b3263eab09], length 0     
08:57:39.796558 IP6 (class 0xc0, flowlabel 0x12753, hlim 2, next-header TCP (6) payload length: 52) ethernetlo0-er1-q8.pnj1.example.net.49047 > bgp.example.com.bgp: Flags [S], cksum 0x9c2d ($
ncorrect -> 0x954f), seq 944299383, win 28800, options [nop,nop,md5 shared secret not supplied with -M, can't check - fd91a8793567cce0b4517caa4d996c43,mss 1440,nop,nop,sackOK,nop,wscale 1$
], length 0                                                 
08:57:40.371415 IP (tos 0x0, ttl 64, id 53428, offset 0, flags [DF], proto TCP (6), length 72)
    bgp.example.com.41025 > 169.254.169.254.bgp: Flags [S], cksum 0x2556 (incorrect -> 0x639a), seq 1962027744, win 29200, options [nop,nop,md5 shared secret not supplied with -M, can't che$
k - b46d10ca7277535792e68799e62d43a9,mss 1460,nop,nop,sackOK,nop,wscale 6], length 0
08:57:40.371513 IP (tos 0xc0, ttl 255, id 0, offset 0, flags [DF], proto TCP (6), length 72)
    169.254.169.254.bgp > bgp.example.com.41025: Flags [S.], cksum 0x2556 (incorrect -> 0xc70a), seq 1444597418, ack 1962027745, win 29200, options [nop,nop,md5 shared secret not supplied w$
th -M, can't check - 7f065ff4f9661c58d59ace759a6e13d9,mss 1460,nop,nop,sackOK,nop,wscale 11], length 0
08:57:40.371538 IP (tos 0x0, ttl 64, id 53429, offset 0, flags [DF], proto TCP (6), length 60)
    bgp.example.com.41025 > 169.254.169.254.bgp: Flags [.], cksum 0x254a (incorrect -> 0xb6a1), seq 1, ack 1, win 457, options [nop,nop,md5 shared secret not supplied with -M, can't check -
4c0a4f7891025b0c28db1acd51f5eb69], length 0   
08:57:40.371914 IP (tos 0xc0, ttl 2, id 63721, offset 0, flags [DF], proto TCP (6), length 149)
    169.254.169.254.bgp > bgp.example.com.41025: Flags [P.], cksum 0x25a3 (incorrect -> 0xd0c8), seq 1:90, ack 1, win 15, options [nop,nop,md5 shared secret not supplied with -M, can't chec$
 - 69a59ad9173d58ca9c1d145c04902182], length 89: BGP        
        Open Message (1), length: 89        
          Version 4, my AS 64515, Holdtime 180s, ID 104.156.240.93.example.com
          Optional parameters, length: 60
            Option Capabilities Advertisement (2), length: 6
              Multiprotocol Extensions (1), length: 4
                AFI IPv4 (1), SAFI Unicast (1)
                0x0000:  0001 0001
            Option Capabilities Advertisement (2), length: 2
              Route Refresh (Cisco) (128), length: 0
            Option Capabilities Advertisement (2), length: 2
              Route Refresh (2), length: 0
            Option Capabilities Advertisement (2), length: 6
              32-Bit AS Number (65), length: 4
                 4 Byte AS 64515                                                                                                                                                            
                0x0000:  0000 fc03                                                                                                                                                          
            Option Capabilities Advertisement (2), length: 6                                  
              Multiple Paths (69), length: 4                                                                                                                                               
                AFI IPv4 (1), SAFI Unicast (1), Send/Receive: Receive                                                                                                                      
                0x0000:  0001 0101                                                             
            Option Capabilities Advertisement (2), length: 20                                                                                                                               
              Unknown (73), length: 18                                                                                                                                                      
                no decoder for Capability 73
                0x0000:  106d 6961 3432 382e 7675 6c74 722e 636f                                                                                                                            
                0x0010:  6d00                                                                                                                                                               
            Option Capabilities Advertisement (2), length: 4
              Graceful Restart (64), length: 2                                                
                Restart Flags: [none], Restart Time 3s                                                                                                                                      
                0x0000:  0003                                                       
08:57:40.371937 IP (tos 0x0, ttl 64, id 53430, offset 0, flags [DF], proto TCP (6), length 60)
    bgp.example.com.41025 > 169.254.169.254.bgp: Flags [.], cksum 0x254a (incorrect -> 0xc093), seq 1, ack 90, win 457, options [nop,nop,md5 shared secret not supplied with -M, can't check -
 7d64613ccce4e609f5abf0c9dd28a91f], length 0                                                          
08:57:40.386321 IP (tos 0x0, ttl 64, id 53431, offset 0, flags [DF], proto TCP (6), length 60)
    bgp.example.com.41025 > 169.254.169.254.bgp: Flags [R.], cksum 0x254a (incorrect -> 0x3494), seq 1, ack 90, win 457, options [nop,nop,md5 shared secret not supplied with -M, can't check
- ed5714ad3178c4ae193ea564b5e51d95], length 0 
08:57:40.818137 IP6 (class 0xc0, flowlabel 0x51ff5, hlim 2, next-header TCP (6) payload length: 52) ethernetlo0-er1-q8.pnj1.example.net.49047 > bgp.example.com.bgp: Flags [S], cksum 0x9c2d (i
ncorrect -> 0x954f), seq 944299383, win 28800, options [nop,nop,md5 shared secret not supplied with -M, can't check - fd91a8793567cce0b4517caa4d996c43,mss 1440,nop,nop,sackOK,nop,wscale 11
], length 0                                                 
08:57:42.866114 IP6 (class 0xc0, flowlabel 0xa2c45, hlim 2, next-header TCP (6) payload length: 52) ethernetlo0-er1-q8.pnj1.example.net.49047 > bgp.example.com.bgp: Flags [S], cksum 0x9c2d (i
ncorrect -> 0x954f), seq 944299383, win 28800, options [nop,nop,md5 shared secret not supplied with -M, can't check - fd91a8793567cce0b4517caa4d996c43,mss 1440,nop,nop,sackOK,nop,wscale 11
], length 0                              
08:57:43.889081 IP (tos 0xc0, ttl 2, id 29244, offset 0, flags [DF], proto TCP (6), length 72)
    169.254.169.254.49091 > bgp.example.com.bgp: Flags [S], cksum 0x2556 (incorrect -> 0xf92f), seq 1087010161, win 29200, options [nop,nop,md5 shared secret not supplied with -M, can't chec
k - 6c1335785efc3ee5c93e58c2cf2ad33e,mss 1460,nop,nop,sackOK,nop,wscale 11], length 0
08:57:46.898145 IP6 (class 0xc0, flowlabel 0xceff8, hlim 2, next-header TCP (6) payload length: 52) ethernetlo0-er1-q8.pnj1.example.net.49047 > bgp.example.com.bgp: Flags [S], cksum 0x9c2d (i
ncorrect -> 0x954f), seq 944299383, win 28800, options [nop,nop,md5 shared secret not supplied with -M, can't check - fd91a8793567cce0b4517caa4d996c43,mss 1440,nop,nop,sackOK,nop,wscale 11
], length 0
thomas-mangin commented 4 years ago

@jkldgoefgkljefogeg as you are using MD5 the tcpdump is not most useful. Can you please provide me the full output of exabgp -d. This is a request in the template to open issue. The configuration you gave me did not include any MD5 information.

thomas-mangin commented 4 years ago

ExaBGP does provide me the data I need to reproduce the issue (the raw OPEN message in a format I can copy and paste using exabgp <your config> --decode <open data>. The dump you provided me is not even a pcap file which I could use. Please provide me what I ask as if I can not reproduce I can not help.

jkldgoefgkljefogeg commented 4 years ago 
$ exabgp test_pipe_post.conf  -d
09:54:59 | 20565  | welcome       | Thank you for using ExaBGP
09:54:59 | 20565  | version       | 4.1.2-d006a34a
09:54:59 | 20565  | interpreter   | 3.8.0 (default, Oct 28 2019, 16:14:01)  [GCC 8.3.0]
09:54:59 | 20565  | os            | Linux server.example.com 4.15.0-66-generic #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019 x86_64
09:54:59 | 20565  | installation  | /home/user/.local
09:54:59 | 20565  | advice        | environment file missing
09:54:59 | 20565  | advice        | generate it using "exabgp --fi > /home/user/.local/etc/exabgp/exabgp.env"
09:54:59 | 20565  | cli           | could not find the named pipes (exabgp.in and exabgp.out) required for the cli
09:54:59 | 20565  | cli           | we scanned the following folders (the number is your PID):
09:54:59 | 20565  | cli control   |  - /run/exabgp/
09:54:59 | 20565  | cli control   |  - /run/1000/
09:54:59 | 20565  | cli control   |  - /run/
09:54:59 | 20565  | cli control   |  - /var/run/exabgp/
09:54:59 | 20565  | cli control   |  - /var/run/1000/
09:54:59 | 20565  | cli control   |  - /var/run/
09:54:59 | 20565  | cli control   |  - /home/user/.local/run/exabgp/
09:54:59 | 20565  | cli control   |  - /home/user/.local/run/1000/
09:54:59 | 20565  | cli control   |  - /home/user/.local/run/
09:54:59 | 20565  | cli control   |  - /home/user/.local/var/run/exabgp/
09:54:59 | 20565  | cli control   |  - /home/user/.local/var/run/1000/
09:54:59 | 20565  | cli control   |  - /home/user/.local/var/run/
09:54:59 | 20565  | cli control   | please make them in one of the folder with the following commands:
09:54:59 | 20565  | cli control   | > mkfifo /home/user/tmp/exabgp/run/exabgp.{in,out}
09:54:59 | 20565  | cli control   | > chmod 600 /home/user/tmp/exabgp/run/exabgp.{in,out}
09:54:59 | 20565  | cli control   | > chown 1000:1000 /home/user/tmp/exabgp/run/exabgp.{in,out}
09:54:59 | 20565  | configuration | performing reload of exabgp 4.1.2-d006a34a
09:54:59 | 20565  | configuration | > process          | 'exabgp_pipe'
09:54:59 | 20565  | configuration | . run              | '/usr/bin/python3.8' '/home/user/tmp/exabgp/exabgp_post.py'
09:54:59 | 20565  | configuration | . encoder          | 'json'
09:54:59 | 20565  | configuration | < process          | 
09:54:59 | 20565  | configuration | > template         | 
09:54:59 | 20565  | configuration | > neighbor         | 'example-v4'
09:54:59 | 20565  | configuration | . router-id        | '203.0.113.209'
09:54:59 | 20565  | configuration | . local-address    | '203.0.113.209'
09:54:59 | 20565  | configuration | . local-as         | '30216'
09:54:59 | 20565  | configuration | > family           | 
09:54:59 | 20565  | configuration | . ipv4             | 'unicast'
09:54:59 | 20565  | configuration | < family           | 
09:54:59 | 20565  | configuration | > api              | 
09:54:59 | 20565  | configuration | . processes        | '[' 'exabgp_pipe' ']'
09:54:59 | 20565  | configuration | . neighbor-changes | 
09:54:59 | 20565  | configuration | . negotiated       | 
09:54:59 | 20565  | configuration | . fsm              | 
09:54:59 | 20565  | configuration | . signal           | 
09:54:59 | 20565  | configuration | > receive          | 
09:54:59 | 20565  | configuration | . parsed           | 
09:54:59 | 20565  | configuration | . packets          | 
09:54:59 | 20565  | configuration | . consolidate      | 
09:54:59 | 20565  | configuration | . open             | 
09:54:59 | 20565  | configuration | . update           | 
09:54:59 | 20565  | configuration | . notification     | 
09:54:59 | 20565  | configuration | . keepalive        | 
09:54:59 | 20565  | configuration | . refresh          | 
09:54:59 | 20565  | configuration | . operational      | 
09:54:59 | 20565  | configuration | < receive          | 
09:54:59 | 20565  | configuration | > send             | 
09:54:59 | 20565  | configuration | . parsed           | 
09:54:59 | 20565  | configuration | . packets          | 
09:54:59 | 20565  | configuration | . consolidate      | 
09:54:59 | 20565  | configuration | . open             | 
09:54:59 | 20565  | configuration | . update           | 
09:54:59 | 20565  | configuration | . notification     | 
09:54:59 | 20565  | configuration | . keepalive        | 
09:54:59 | 20565  | configuration | . refresh          | 
09:54:59 | 20565  | configuration | . operational      | 
09:54:59 | 20565  | configuration | < send             | 
09:54:59 | 20565  | configuration | < api              | 
09:54:59 | 20565  | configuration | < neighbor         | 
09:54:59 | 20565  | configuration | < template         | 
09:54:59 | 20565  | configuration | > neighbor         | '169.254.169.254'
09:54:59 | 20565  | configuration | . inherit          | 'example-v4'
09:54:59 | 20565  | configuration | . description      | 'vultr-v4'
09:54:59 | 20565  | configuration | . peer-as          | '64515'
09:54:59 | 20565  | configuration | . md5-password     | '**********'
09:54:59 | 20565  | configuration | < neighbor         | 
09:54:59 | 20565  | reactor       | new peer: neighbor 169.254.169.254 local-ip 203.0.113.209 local-as 30216 peer-as 64515 router-id 203.0.113.209 family-allowed in-open
09:54:59 | 20565  | reactor       | loaded new configuration successfully
09:54:59 | 20565  | process       | forked process exabgp_pipe
09:54:59 | 20565  | reactor       | initialising connection to peer-1
09:54:59 | 20565  | outgoing-1    | attempting connection to 169.254.169.254:179
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       | ********************************************************************************
09:54:59 | 20565  | reactor       | EXABGP MISBEHAVED / HELP US FIX IT
09:54:59 | 20565  | reactor       | ********************************************************************************
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       | Sorry, you encountered a problem with ExaBGP, as the problem only affects one
09:54:59 | 20565  | reactor       | peer, we are trying to keep the program running.
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       | There are a few things you can do to help us (and yourself):
09:54:59 | 20565  | reactor       | - make sure you are running the latest version of the code available at
09:54:59 | 20565  | reactor       |   https://github.com/Exa-Networks/exabgp/releases/latest
09:54:59 | 20565  | reactor       | - if so report the issue on https://github.com/Exa-Networks/exabgp/issues
09:54:59 | 20565  | reactor       |   so it can be fixed (github can be searched for similar reports)
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       | PLEASE, when reporting, do include as much information as you can:
09:54:59 | 20565  | reactor       | - do not obfuscate any data (feel free to send us a private  email with the
09:54:59 | 20565  | reactor       |   extra information if your business policy is strict on information sharing)
09:54:59 | 20565  | reactor       |   https://github.com/Exa-Networks/exabgp/wiki/FAQ
09:54:59 | 20565  | reactor       | - if you can reproduce the issue, run ExaBGP with the command line option -d
09:54:59 | 20565  | reactor       |   it provides us with much needed information to fix problems quickly
09:54:59 | 20565  | reactor       | - include the information presented below
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       | Should you not receive an acknowledgment of your issue on github (assignement,
09:54:59 | 20565  | reactor       | comment, or similar) within a few hours, feel free to email us to make sure
09:54:59 | 20565  | reactor       | it was not overlooked. (please keep in mind the authors are based in GMT/Europe)
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       | ********************************************************************************
09:54:59 | 20565  | reactor       | -- Please provide ALL the information below on :
09:54:59 | 20565  | reactor       | -- https://github.com/Exa-Networks/exabgp/issues
09:54:59 | 20565  | reactor       | ********************************************************************************
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       | ExaBGP version : 4.1.2-d006a34a
09:54:59 | 20565  | reactor       | Python version : 3.8.0 (default, Oct 28 2019, 16:14:01)  [GCC 8.3.0]
09:54:59 | 20565  | reactor       | System Uname   : #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019
09:54:59 | 20565  | reactor       | System MaxInt  : 9223372036854775807
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       | <class 'TypeError'>
09:54:59 | 20565  | reactor       | unhashable type: 'Neighbor'
09:54:59 | 20565  | reactor       | Traceback (most recent call last):
09:54:59 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/peer.py", line 545, in _run
09:54:59 | 20565  | reactor       |     for action in self._establish():
09:54:59 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/peer.py", line 343, in _establish
09:54:59 | 20565  | reactor       |     for sent_open in self._send_open():
09:54:59 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/peer.py", line 299, in _send_open
09:54:59 | 20565  | reactor       |     for message in self.proto.new_open():
09:54:59 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/protocol.py", line 318, in new_open
09:54:59 | 20565  | reactor       |     for _ in self.write(sent_open):
09:54:59 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/protocol.py", line 165, in write
09:54:59 | 20565  | reactor       |     self._to_api('send',message,raw)
09:54:59 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/protocol.py", line 150, in _to_api
09:54:59 | 20565  | reactor       |     self.peer.reactor.processes.message(self.peer.neighbor,direction,message,negotiated,raw[:19],raw[19:])
09:54:59 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/api/processes.py", line 292, in closure
09:54:59 | 20565  | reactor       |     return function(self,*args)
09:54:59 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/api/processes.py", line 338, in message
09:54:59 | 20565  | reactor       |     self._dispatch[message_id](self,neighbor,direction,message,negotiated,header,*body)
09:54:59 | 20565  | reactor       | TypeError: unhashable type: 'Neighbor'
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       | ********************************************************************************
09:54:59 | 20565  | reactor       | -- Please provide _ALL_ the information above on :
09:54:59 | 20565  | reactor       | -- https://github.com/Exa-Networks/exabgp/issues
09:54:59 | 20565  | reactor       | ********************************************************************************
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | outgoing-1    | peer reset, message [] error[]
09:54:59 | 20565  | outgoing-1    | outgoing-1 203.0.113.209-169.254.169.254, closing connection
09:54:59 | 20565  | reactor       | initialising connection to peer-1
09:54:59 | 20565  | outgoing-2    | attempting connection to 169.254.169.254:179
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       | ********************************************************************************
09:54:59 | 20565  | reactor       | EXABGP MISBEHAVED / HELP US FIX IT
09:54:59 | 20565  | reactor       | ********************************************************************************
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       | Sorry, you encountered a problem with ExaBGP, as the problem only affects one
09:54:59 | 20565  | reactor       | peer, we are trying to keep the program running.
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       | There are a few things you can do to help us (and yourself):
09:54:59 | 20565  | reactor       | - make sure you are running the latest version of the code available at
09:54:59 | 20565  | reactor       |   https://github.com/Exa-Networks/exabgp/releases/latest
09:54:59 | 20565  | reactor       | - if so report the issue on https://github.com/Exa-Networks/exabgp/issues
09:54:59 | 20565  | reactor       |   so it can be fixed (github can be searched for similar reports)
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       | PLEASE, when reporting, do include as much information as you can:
09:54:59 | 20565  | reactor       | - do not obfuscate any data (feel free to send us a private  email with the
09:54:59 | 20565  | reactor       |   extra information if your business policy is strict on information sharing)
09:54:59 | 20565  | reactor       |   https://github.com/Exa-Networks/exabgp/wiki/FAQ
09:54:59 | 20565  | reactor       | - if you can reproduce the issue, run ExaBGP with the command line option -d
09:54:59 | 20565  | reactor       |   it provides us with much needed information to fix problems quickly
09:54:59 | 20565  | reactor       | - include the information presented below
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       | Should you not receive an acknowledgment of your issue on github (assignement,
09:54:59 | 20565  | reactor       | comment, or similar) within a few hours, feel free to email us to make sure
09:54:59 | 20565  | reactor       | it was not overlooked. (please keep in mind the authors are based in GMT/Europe)
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       | ********************************************************************************
09:54:59 | 20565  | reactor       | -- Please provide ALL the information below on :
09:54:59 | 20565  | reactor       | -- https://github.com/Exa-Networks/exabgp/issues
09:54:59 | 20565  | reactor       | ********************************************************************************
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       | ExaBGP version : 4.1.2-d006a34a
09:54:59 | 20565  | reactor       | Python version : 3.8.0 (default, Oct 28 2019, 16:14:01)  [GCC 8.3.0]
09:54:59 | 20565  | reactor       | System Uname   : #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019
09:54:59 | 20565  | reactor       | System MaxInt  : 9223372036854775807
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       | <class 'TypeError'>
09:54:59 | 20565  | reactor       | unhashable type: 'Neighbor'
09:54:59 | 20565  | reactor       | Traceback (most recent call last):
09:54:59 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/peer.py", line 545, in _run
09:54:59 | 20565  | reactor       |     for action in self._establish():
09:54:59 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/peer.py", line 343, in _establish
09:54:59 | 20565  | reactor       |     for sent_open in self._send_open():
09:54:59 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/peer.py", line 299, in _send_open
09:54:59 | 20565  | reactor       |     for message in self.proto.new_open():
09:54:59 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/protocol.py", line 318, in new_open
09:54:59 | 20565  | reactor       |     for _ in self.write(sent_open):
09:54:59 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/protocol.py", line 165, in write
09:54:59 | 20565  | reactor       |     self._to_api('send',message,raw)
09:54:59 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/protocol.py", line 150, in _to_api
09:54:59 | 20565  | reactor       |     self.peer.reactor.processes.message(self.peer.neighbor,direction,message,negotiated,raw[:19],raw[19:])
09:54:59 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/api/processes.py", line 292, in closure
09:54:59 | 20565  | reactor       |     return function(self,*args)
09:54:59 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/api/processes.py", line 338, in message
09:54:59 | 20565  | reactor       |     self._dispatch[message_id](self,neighbor,direction,message,negotiated,header,*body)
09:54:59 | 20565  | reactor       | TypeError: unhashable type: 'Neighbor'
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | reactor       | ********************************************************************************
09:54:59 | 20565  | reactor       | -- Please provide _ALL_ the information above on :
09:54:59 | 20565  | reactor       | -- https://github.com/Exa-Networks/exabgp/issues
09:54:59 | 20565  | reactor       | ********************************************************************************
09:54:59 | 20565  | reactor       |         
09:54:59 | 20565  | outgoing-2    | peer reset, message [] error[]
09:54:59 | 20565  | outgoing-2    | outgoing-2 203.0.113.209-169.254.169.254, closing connection
09:55:00 | 20565  | reactor       | initialising connection to peer-1
09:55:00 | 20565  | outgoing-3    | attempting connection to 169.254.169.254:179
09:55:00 | 20565  | reactor       |         
09:55:00 | 20565  | reactor       | ********************************************************************************
09:55:00 | 20565  | reactor       | EXABGP MISBEHAVED / HELP US FIX IT
09:55:00 | 20565  | reactor       | ********************************************************************************
09:55:00 | 20565  | reactor       |         
09:55:00 | 20565  | reactor       | Sorry, you encountered a problem with ExaBGP, as the problem only affects one
09:55:00 | 20565  | reactor       | peer, we are trying to keep the program running.
09:55:00 | 20565  | reactor       |         
09:55:00 | 20565  | reactor       | There are a few things you can do to help us (and yourself):
09:55:00 | 20565  | reactor       | - make sure you are running the latest version of the code available at
09:55:00 | 20565  | reactor       |   https://github.com/Exa-Networks/exabgp/releases/latest
09:55:00 | 20565  | reactor       | - if so report the issue on https://github.com/Exa-Networks/exabgp/issues
09:55:00 | 20565  | reactor       |   so it can be fixed (github can be searched for similar reports)
09:55:00 | 20565  | reactor       |         
09:55:00 | 20565  | reactor       | PLEASE, when reporting, do include as much information as you can:
09:55:00 | 20565  | reactor       | - do not obfuscate any data (feel free to send us a private  email with the
09:55:00 | 20565  | reactor       |   extra information if your business policy is strict on information sharing)
09:55:00 | 20565  | reactor       |   https://github.com/Exa-Networks/exabgp/wiki/FAQ
09:55:00 | 20565  | reactor       | - if you can reproduce the issue, run ExaBGP with the command line option -d
09:55:00 | 20565  | reactor       |   it provides us with much needed information to fix problems quickly
09:55:00 | 20565  | reactor       | - include the information presented below
09:55:00 | 20565  | reactor       |         
09:55:00 | 20565  | reactor       | Should you not receive an acknowledgment of your issue on github (assignement,
09:55:00 | 20565  | reactor       | comment, or similar) within a few hours, feel free to email us to make sure
09:55:00 | 20565  | reactor       | it was not overlooked. (please keep in mind the authors are based in GMT/Europe)
09:55:00 | 20565  | reactor       |         
09:55:00 | 20565  | reactor       | ********************************************************************************
09:55:00 | 20565  | reactor       | -- Please provide ALL the information below on :
09:55:00 | 20565  | reactor       | -- https://github.com/Exa-Networks/exabgp/issues
09:55:00 | 20565  | reactor       | ********************************************************************************
09:55:00 | 20565  | reactor       |         
09:55:00 | 20565  | reactor       | ExaBGP version : 4.1.2-d006a34a
09:55:00 | 20565  | reactor       | Python version : 3.8.0 (default, Oct 28 2019, 16:14:01)  [GCC 8.3.0]
09:55:00 | 20565  | reactor       | System Uname   : #75-Ubuntu SMP Tue Oct 1 05:24:09 UTC 2019
09:55:00 | 20565  | reactor       | System MaxInt  : 9223372036854775807
09:55:00 | 20565  | reactor       |         
09:55:00 | 20565  | reactor       |         
09:55:00 | 20565  | reactor       |         
09:55:00 | 20565  | reactor       |         
09:55:00 | 20565  | reactor       | <class 'TypeError'>
09:55:00 | 20565  | reactor       | unhashable type: 'Neighbor'
09:55:00 | 20565  | reactor       | Traceback (most recent call last):
09:55:00 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/peer.py", line 545, in _run
09:55:00 | 20565  | reactor       |     for action in self._establish():
09:55:00 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/peer.py", line 343, in _establish
09:55:00 | 20565  | reactor       |     for sent_open in self._send_open():
09:55:00 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/peer.py", line 299, in _send_open
09:55:00 | 20565  | reactor       |     for message in self.proto.new_open():
09:55:00 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/protocol.py", line 318, in new_open
09:55:00 | 20565  | reactor       |     for _ in self.write(sent_open):
09:55:00 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/protocol.py", line 165, in write
09:55:00 | 20565  | reactor       |     self._to_api('send',message,raw)
09:55:00 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/protocol.py", line 150, in _to_api
09:55:00 | 20565  | reactor       |     self.peer.reactor.processes.message(self.peer.neighbor,direction,message,negotiated,raw[:19],raw[19:])
09:55:00 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/api/processes.py", line 292, in closure
09:55:00 | 20565  | reactor       |     return function(self,*args)
09:55:00 | 20565  | reactor       |   File "/home/user/.local/lib/python3.8/site-packages/exabgp/reactor/api/processes.py", line 338, in message
09:55:00 | 20565  | reactor       |     self._dispatch[message_id](self,neighbor,direction,message,negotiated,header,*body)
09:55:00 | 20565  | reactor       | TypeError: unhashable type: 'Neighbor'
09:55:00 | 20565  | reactor       |         
09:55:00 | 20565  | reactor       | ********************************************************************************
09:55:00 | 20565  | reactor       | -- Please provide _ALL_ the information above on :
09:55:00 | 20565  | reactor       | -- https://github.com/Exa-Networks/exabgp/issues
09:55:00 | 20565  | reactor       | ********************************************************************************
09:55:00 | 20565  | reactor       |         
09:55:00 | 20565  | outgoing-3    | peer reset, message [] error[]
thomas-mangin commented 4 years ago

Thank you - I will look into it now.

thomas-mangin commented 4 years ago 
09:54:59 | 20565  | configuration | . router-id        | '198.51.100..209'
09:54:59 | 20565  | configuration | . local-address    | '198.51.100..209'
        Open Message (1), length: 89
          Version 4, my AS 64515, Holdtime 180s, ID 104.156.240.93.example.com

The IP are not matching .. 198.51.100..209 .. You are really making it hard for me to help you. And I am not even sure how ExaBGP can validate the ".." so you are editing the data you are sending.

Please provide me the output of "tcpdump -s0 -p -w file.pcap -i any '' port 179" so I can extract the raw open message myself.

jkldgoefgkljefogeg commented 4 years ago

updated with original -d output, sorry about that

thomas-mangin commented 4 years ago

This bug is fixed. The configuration you gave me is not the one which generated this open. In the configuration you used you have some OPEN capability setup which is not in what you provided me.

jkldgoefgkljefogeg commented 4 years ago

send consolidate + packet no longer crashes, but packet without consolidate still does

This is still missing message.ID? https://github.com/Exa-Networks/exabgp/blob/master/lib/exabgp/reactor/protocol.py#L153

How I triggered it

                api {
                        processes [exabgp_pipe];
                        neighbor-changes;
                        negotiated;
                        fsm;
                        signal;
                        receive {
                                parsed;
                                packets;
                                consolidate;
                                open;
                                update;
                                notification;
                                keepalive;
                                refresh;
                                operational;
                        }
                        send {
                                parsed;
                                packets;
                        #       consolidate;
                                open;
                                update;
                                notification;
                                keepalive;
                                refresh;
                                operational;
                        }
                }

        }
}
22:21:04 | 3532   | reactor       | <class 'TypeError'>
22:21:04 | 3532   | reactor       | packets() takes 6 positional arguments but 7 were given
22:21:04 | 3532   | reactor       | Traceback (most recent call last):
22:21:04 | 3532   | reactor       |   File "/home/user/git/exabgp-git/lib/exabgp/reactor/peer.py", line 540, in _run
22:21:04 | 3532   | reactor       |     for action in self._establish():
22:21:04 | 3532   | reactor       |   File "/home/user/git/exabgp-git/lib/exabgp/reactor/peer.py", line 337, in _establish
22:21:04 | 3532   | reactor       |     for sent_open in self._send_open():
22:21:04 | 3532   | reactor       |   File "/home/user/git/exabgp-git/lib/exabgp/reactor/peer.py", line 293, in _send_open
22:21:04 | 3532   | reactor       |     for message in self.proto.new_open():
22:21:04 | 3532   | reactor       |   File "/home/user/git/exabgp-git/lib/exabgp/reactor/protocol.py", line 316, in new_open
22:21:04 | 3532   | reactor       |     for _ in self.write(sent_open):
22:21:04 | 3532   | reactor       |   File "/home/user/git/exabgp-git/lib/exabgp/reactor/protocol.py", line 163, in write
22:21:04 | 3532   | reactor       |     self._to_api('send',message,raw)
22:21:04 | 3532   | reactor       |   File "/home/user/git/exabgp-git/lib/exabgp/reactor/protocol.py", line 153, in _to_api
22:21:04 | 3532   | reactor       |     self.peer.reactor.processes.packets(self.peer.neighbor,direction,int(message.ID),negotiated,raw[:19],raw[19:])
22:21:04 | 3532   | reactor       |   File "/home/user/git/exabgp-git/lib/exabgp/reactor/api/processes.py", line 329, in closure
22:21:04 | 3532   | reactor       |     return function(self,*args)
22:21:04 | 3532   | reactor       | TypeError: packets() takes 6 positional arguments but 7 were given
thomas-mangin commented 4 years ago

fixed too.

jkldgoefgkljefogeg commented 4 years ago

Thanks. Confirmed it is now working with consolidate/packets