next-hop not being parsed in BGP Junos

elkinaguas commented 6 years ago

Hello, I think I found a problem with next-hop on Junos. I want to announce a static route on BGP, so I configure it inside the routing-options section, here is how it looks:

routing-options {                       
    static {                            
        route 192.168.4.0/24 next-hop 192.168.4.0;
    }                                   
    autonomous-system 200;              
}

The idea is to populate a network-instance model from the device, change the config and then merge the new configuration back to the device. When doing this I get this error.

Traceback (most recent call last):
  File "openConfigEx.py", line 79, in <module>
    d.load_merge_candidate(config=config2)
  File "/usr/local/lib/python2.7/dist-packages/napalm/junos/junos.py", line 227, in load_merge_candidate
    self._load_candidate(filename, config, False)
  File "/usr/local/lib/python2.7/dist-packages/napalm/junos/junos.py", line 217, in _load_candidate
    raise MergeConfigException(e.errs)
napalm.base.exceptions.MergeConfigException: [{'source': None, 'message': 'mgd: statement has no contents; ignored', 'bad_element': 'route 192.168.4.0/24', 'severity': 'warning', 'edit_path': '[edit routing-options static]'}]

I started getting this error when adding the static route in routing-options. I noticed as well that the configuration parsed from the device does not contain the next-hop information, this can be seen bellow in the cofig parsed from the device as well as in the translation of this config to Junos native configuration.

I'm working with a Junos 12.1 VM and napalm-yang version 0.1.0.

Device configuration

## Last commit: 2018-06-21 10:23:11 UTC by root
version 12.1X47-D15.4;
system {
    host-name r2;
    root-authentication {
        encrypted-password "$1$nq.N1UsY$JxA/ESAj3KuXseXE597gg0"; ## SECRET-DATA
        ssh-rsa "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCzFChCRM5cviOavo6Bta1gUHjdWraB9ZLWAwPhkSr9lkj8qqfLpvxch7ADsL5AUO1cox6Se9oyWRZQutEbZTW+pYPnR7W05LzbVpDURfc85BuYOHWVdWUjiB764fslEUvH+0NkeAWO/Hag6AKPYluATGM3vkRZLjPxNLtR4D0M5rfN3EmvBq1EhY8OKhBY+ELxmny0UW+hj69IyVEwD5WU/eo5OYWr2JbEBJ2TA7T4YQ8bQ0ZaVu1G5CGIR/NH8TOBh/4BhNWhj4/knSgyU+nupRvgjSf8dFspSN68udhCxByJerXFb2suwJuVsb9TN6yrAsUYqoDjphwjnQPC8Erb vagrant"; ## SECRET-DATA
    }
    login {
        user vagrant {
            uid 2000;
            class super-user;
            authentication {
                ssh-rsa "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzIw+niNltGEFHzD8+v1I2YJ6oXevct1YeS0o9HZyN1Q9qgCgzUFtdOKLv6IedplqoPkcmF0aYet2PkEDo3MlTBckFXPITAMzF8dJSIFo9D8HfdOV0IAdx4O7PtixWKn5y2hMNG0zQPyUecp4pzC6kivAIhyfHilFR61RGL+GPXQ2MWZWFYbAGjyiYJnAmCP3NOTd0jMZEnDkbUvxhMmBYSdETk1rRgm+R4LOzFUGaHqHDLKLX+FIPKcF96hrucXzcWyLbIbEgE98OHlnVYCzRdK8jlqm8tehUc9c9WhQ== vagrant insecure public key"; ## SECRET-DATA
            }                           
        }                               
    }                                   
    services {                          
        ssh {                           
            root-login allow;           
        }                               
        netconf {                       
            ssh;                        
        }                               
        web-management {                
            http {                      
                interface ge-0/0/0.0;   
            }                           
        }                               
    }                                   
    syslog {                            
        user * {                        
            any emergency;              
        }                               
        file messages {                 
            any any;                    
            authorization info;         
        }                               
        file interactive-commands {     
            interactive-commands any;   
        }                               
    }                                   
    license {                           
        autoupdate {                    
            url https://ae1.juniper.net/junos/key_retrieval;
        }                               
    }                                   
}                                       
interfaces {                            
    ge-0/0/0 {                          
        unit 0 {                        
            family inet {               
                dhcp;                   
            }                           
        }                               
    }                                   
    ge-0/0/1 {                          
        unit 0 {                        
            family inet {               
                address 192.168.1.2/24; 
            }                           
        }                               
    }                                   
    ge-0/0/2 {                          
        unit 0 {                        
            family inet {               
                address 192.168.4.1/24; 
            }                           
        }                               
    }                                   
    lo0 {                               
        unit 0 {                        
            family inet {               
                address 2.2.2.2/32;     
            }                           
        }                               
    }                                   
}                                       
routing-options {                       
    static {                            
        route 192.168.4.0/24 next-hop 192.168.4.0;
    }                                   
    autonomous-system 200;              
}                                       
protocols {                             
    bgp {                               
        group external-peers {          
            type external;              
            export send-direct;         
            neighbor 192.168.1.1 {      
                peer-as 100;            
            }                           
        }                               
    }                                   
}                                       
policy-options {                        
    policy-statement send-direct {      
        term accept {                   
            from protocol [ static bgp direct ];
            then accept;                
        }                               
        term reject {                   
            then reject;                
        }                               
    }                                   
}                                       
security {                              
    forwarding-options {                
        family {                        
            inet6 {                     
                mode packet-based;      
            }                           
            mpls {                      
                mode packet-based;      
            }                           
        }                               
    }                                   
}

Configuration parsed

{
    "network_instances": {
        "network-instance": {
            "global": {
                "config": {
                    "enabled": True, 
                    "type": "DEFAULT_INSTANCE"
                }, 
                "name": "global", 
                "protocols": {
                    "protocol": {
                        "bgp bgp": {
                            "bgp": {
                                "global": {
                                    "config": {
                                        "as": 200
                                    }
                                }, 
                                "neighbors": {
                                    "neighbor": {
                                        "192.168.1.1": {
                                            "config": {
                                                "neighbor-address": "192.168.1.1", 
                                                "peer-as": 100, 
                                                "peer-group": "external-peers"
                                            }, 
                                            "neighbor-address": "192.168.1.1"
                                        }
                                    }
                                }
                            }, 
                            "identifier": "bgp", 
                            "name": "bgp"
                        }, 
                        "static static": {
                            "identifier": "static", 
                            "name": "static", 
                            "static-routes": {
                                "static": {
                                    "192.168.4.0/24": {
                                        "config": {
                                            "prefix": "192.168.4.0/24"
                                        }, 
                                        "prefix": "192.168.4.0/24"
                                    }
                                }
                            }
                        }
                    }
                }
            }
        }
    }
}

Configuration translation to Junos

<configuration>
  <protocols>
    <bgp>
      <group>
        <name>external-peers</name>
        <neighbor>
          <name>192.168.1.1</name>
          <peer-as>100</peer-as>
        </neighbor>
      </group>
      <group>
        <name>external-peers</name>
        <neighbor>
          <name>10.100.100.4</name>
          <peer-as>100</peer-as>
        </neighbor>
      </group>
    </bgp>
  </protocols>
  <routing-options>
    <autonomous-system>
      <as-number>200</as-number>
    </autonomous-system>
  </routing-options>
  <routing-options>
    <static>
      <route>
        <name>192.168.4.0/24</name>
      </route>
    </static>
  </routing-options>
</configuration>

My code

from napalm_base import get_network_driver
import napalm_yang, salt.client, ast

import json

def use_real_devices():
    junos_configuration = {
        'hostname': '127.0.0.1',
        'username': 'root',
        'password': 'Juniper',
        'optional_args': {'port': 2232}
    }

    eos_configuration = {
        'hostname': '127.0.0.1',
        'username': 'vagrant',
        'password': 'vagrant',
        'optional_args': {'port': 12443}
    }

    junos = get_network_driver("junos")
    junos_device = junos(**junos_configuration)

    eos = get_network_driver("eos")
    eos_device = eos(**eos_configuration)
    return junos_device, eos_device

def pretty_print(dictionary):
    print(json.dumps(dictionary, sort_keys=True, indent=4))

junos_device, eos_device = use_real_devices()

#__________________________________________________________________________________

with junos_device as d:
    candidate = napalm_yang.base.Root()
    candidate.add_model(napalm_yang.models.openconfig_network_instance)
    candidate.parse_config(device=d)
    pretty_print(candidate.get(filter=True))

    neighbor = candidate.network_instances.network_instance["global"].protocols.protocol["bgp bgp"].bgp.neighbors.neighbor.add("10.100.100.4")

    neighbor.config.neighbor_address = "10.100.100.4"
    neighbor.config.peer_as = 100
    neighbor.config.peer_group = "external-peers"
    pretty_print(candidate.get(filter=True))

    running = napalm_yang.base.Root()
    running.add_model(napalm_yang.models.openconfig_network_instance)
    running.parse_config(device=d)

    config2 = candidate.translate_config(profile=junos_device.profile, merge=running)
    print config2
    d.load_merge_candidate(config=config2)
    print(d.compare_config())
    d.discard_config()

dbarrosop commented 6 years ago

Junos is anything but consistent...

current parser is looking for "qualified-next-hop":

https://github.com/napalm-automation/napalm-yang/blob/develop/napalm_yang/mappings/junos/parsers/config/openconfig-network-instance/includes/static_routes.yaml#L19-L21

Try replacing those three lines with:

            _process:
              - path: "qualified-next-hop"
                key: name
              - path: "next-hop"
                key: name

elkinaguas commented 6 years ago

Totally right about Junos @dbarrosop.

I tried changing the lines, I get the following error.

Traceback (most recent call last):
  File "testNextHop.py", line 38, in <module>
    candidate.parse_config(device=d)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/base.py", line 247, in parse_config
    parser.parse()
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 111, in parse
    self._parse(self._yang_name, self.model, self.mapping[self._yang_name])
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 117, in _parse
    self._parse_container(attribute, model, mapping)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 172, in _parse_container
    self._parse(k, v, mapping[v._yang_name])
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 119, in _parse
    self._parse_list(attribute, model, mapping)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 217, in _parse_list
    self._parse(key, obj, element_mapping)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 117, in _parse
    self._parse_container(attribute, model, mapping)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 172, in _parse_container
    self._parse(k, v, mapping[v._yang_name])
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 117, in _parse
    self._parse_container(attribute, model, mapping)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 172, in _parse_container
    self._parse(k, v, mapping[v._yang_name])
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 119, in _parse
    self._parse_list(attribute, model, mapping)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 217, in _parse_list
    self._parse(key, obj, element_mapping)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 117, in _parse
    self._parse_container(attribute, model, mapping)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 172, in _parse_container
    self._parse(k, v, mapping[v._yang_name])
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 117, in _parse
    self._parse_container(attribute, model, mapping)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 172, in _parse_container
    self._parse(k, v, mapping[v._yang_name])
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 119, in _parse
    self._parse_list(attribute, model, mapping)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 217, in _parse_list
    self._parse(key, obj, element_mapping)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 117, in _parse
    self._parse_container(attribute, model, mapping)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 172, in _parse_container
    self._parse(k, v, mapping[v._yang_name])
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 117, in _parse
    self._parse_container(attribute, model, mapping)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 172, in _parse_container
    self._parse(k, v, mapping[v._yang_name])
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 119, in _parse
    self._parse_list(attribute, model, mapping)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parser.py", line 190, in _parse_list
    attribute, mapping["_process"], self.bookmarks
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parsers/base.py", line 146, in parse_list
    for key, block, extra_vars in self._parse_list_default(attribute, m, data):
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parsers/jsonp.py", line 81, in _parse_list_default
    for k, v in _iterator(d, mapping.get("key")):
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parsers/jsonp.py", line 53, in _iterator
    k = _eval_key(key_mapping, **v)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parsers/jsonp.py", line 41, in _eval_key
    return get_element_with_cdata(kwargs, key_mapping)
  File "/usr/local/lib/python2.7/dist-packages/napalm_yang/parsers/jsonp.py", line 12, in get_element_with_cdata
    e = dictionary[element]
KeyError: 'name'

dbarrosop commented 6 years ago

Can you share the XML configuration? That's what's napalm-yang is parsing.

elkinaguas commented 6 years ago

XML configuration

<rpc-reply xmlns:junos="http://xml.juniper.net/junos/12.1X47/junos">
    <configuration junos:commit-seconds="1530104378" junos:commit-localtime="2018-06-27 12:59:38 UTC" junos:commit-user="root">
            <version>12.1X47-D15.4</version>
            <system>
                <host-name>r2</host-name>
                <root-authentication>
                    <encrypted-password>$1$nq.N1UsY$JxA/ESAj3KuXseXE597gg0</encrypted-password>
                    <ssh-rsa>
                        <name>ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCzFChCRM5cviOavo6Bta1gUHjdWraB9ZLWAwPhkSr9lkj8qqfLpvxch7ADsL5AUO1cox6Se9oyWRZQutEbZTW+pYPnR7W05LzbVpDURfc85BuYOHWVdWUjiB764fslEUvH+0NkeAWO/Hag6AKPYluATGM3vkRZLjPxNLtR4D0M5rfN3EmvBq1EhY8OKhBY+ELxmny0UW+hj69IyVEwD5WU/eo5OYWr2JbEBJ2TA7T4YQ8bQ0ZaVu1G5CGIR/NH8TOBh/4BhNWhj4/knSgyU+nupRvgjSf8dFspSN68udhCxByJerXFb2suwJuVsb9TN6yrAsUYqoDjphwjnQPC8Erb vagrant</name>
                    </ssh-rsa>
                </root-authentication>
                <login>
                    <user>
                        <name>vagrant</name>
                        <uid>2000</uid>
                        <class>super-user</class>
                        <authentication>
                            <ssh-rsa>   
                                <name>ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA6NF8iallvQVp22WDkTkyrtvp9eWW6A8YVr+kz4TjGYe7gHzIw+niNltGEFHzD8+v1I2YJ6oXevct1YeS0o9HZyN1Q9qgCgzUFtdOKLv6IedplqoPkcmF0aYet2PkEDo3MlTBckFXPITAMzF8dJSIFo9D8HfdOV0IAdx4O7PtixWKn5y2hMNG0zQPyUecp4pzC6kivAIhyfHilFR61RGL+GPXQ2MWZWFYbAGjyiYJnAmCP3NOTd0jMZEnDkbUvxhMmBYSdETk1rRgm+R4LOzFUGaHqHDLKLX+FIPKcF96hrucXzcWyLbIbEgE98OHlnVYCzRdK8jlqm8tehUc9c9WhQ== vagrant insecure public key</name>
                            </ssh-rsa>  
                        </authentication>
                    </user>             
                </login>                
                <services>              
                    <ssh>               
                        <root-login>allow</root-login>
                    </ssh>              
                    <netconf>           
                        <ssh>           
                        </ssh>          
                    </netconf>          
                    <web-management>    
                        <http>          
                            <interface>ge-0/0/0.0</interface>
                        </http>         
                    </web-management>   
                </services>             
                <syslog>                
                    <user>              
                        <name>*</name>  
                        <contents>      
                            <name>any</name>
                            <emergency/>
                        </contents>     
                    </user>             
                    <file>              
                        <name>messages</name>
                        <contents>      
                            <name>any</name>
                            <any/>      
                        </contents>     
                        <contents>      
                            <name>authorization</name>
                            <info/>     
                        </contents>     
                    </file>             
                    <file>              
                        <name>interactive-commands</name>
                        <contents>      
                            <name>interactive-commands</name>
                            <any/>      
                        </contents>     
                    </file>             
                </syslog>               
                <license>               
                    <autoupdate>        
                        <url>           
                            <name>https://ae1.juniper.net/junos/key_retrieval</name>
                        </url>          
                    </autoupdate>       
                </license>              
            </system>                   
            <interfaces>                
                <interface>             
                    <name>ge-0/0/0</name>
                    <unit>              
                        <name>0</name>  
                        <family>        
                            <inet>      
                                <dhcp>  
                                </dhcp> 
                            </inet>     
                        </family>       
                    </unit>             
                </interface>            
                <interface>             
                    <name>ge-0/0/1</name>
                    <unit>              
                        <name>0</name>  
                        <family>        
                            <inet>      
                                <address>
                                    <name>192.168.1.2/24</name>
                                </address>
                            </inet>     
                        </family>       
                    </unit>             
                </interface>            
                <interface>             
                    <name>ge-0/0/2</name>
                    <unit>              
                        <name>0</name>  
                        <family>        
                            <inet>      
                                <address>
                                    <name>192.168.4.1/24</name>
                                </address>
                            </inet>     
                        </family>       
                    </unit>             
                </interface>            
                <interface>             
                    <name>lo0</name>    
                    <unit>              
                        <name>0</name>  
                        <family>        
                            <inet>      
                                <address>
                                    <name>2.2.2.2/32</name>
                                </address>
                            </inet>     
                        </family>       
                    </unit>             
                </interface>            
            </interfaces>               
            <routing-options>           
                <static>                
                    <route>             
                        <name>192.168.4.0/24</name>
                        <next-hop>192.168.4.0</next-hop>
                    </route>            
                </static>               
                <autonomous-system>     
                    <as-number>200</as-number>
                </autonomous-system>    
            </routing-options>          
            <protocols>                 
                <bgp>                   
                    <group>             
                        <name>external-peers</name>
                        <type>external</type>
                        <export>send-direct</export>
                        <neighbor>      
                            <name>192.168.1.1</name>
                            <peer-as>100</peer-as>
                        </neighbor>     
                    </group>            
                </bgp>                  
            </protocols>                
            <policy-options>            
                <policy-statement>      
                    <name>send-direct</name>
                    <term>              
                        <name>accept</name>
                        <from>          
                            <protocol>static</protocol>
                            <protocol>bgp</protocol>
                            <protocol>direct</protocol>
                        </from>         
                        <then>          
                            <accept/>   
                        </then>         
                    </term>             
                    <term>              
                        <name>reject</name>
                        <then>          
                            <reject/>   
                        </then>         
                    </term>             
                </policy-statement>     
            </policy-options>           
            <security>                  
                <forwarding-options>    
                    <family>            
                        <inet6>         
                            <mode>packet-based</mode>
                        </inet6>        
                        <mpls>          
                            <mode>packet-based</mode>
                        </mpls>         
                    </family>           
                </forwarding-options>   
            </security>                 
    </configuration>                    
    <cli>                               
        <banner></banner>               
    </cli>                              
</rpc-reply>

dbarrosop commented 6 years ago

Ok, this is very odd. I have an image with "12.1X47-D20.7" and everything works fine. The difference is as follows:

            <static>
                <route>
                    <name>10.0.0.0/24</name>
                    <qualified-next-hop>
                        <name>192.168.100.100</name>
                        <metric>100</metric>
                    </qualified-next-hop>
                    <qualified-next-hop>
                        <name>192.168.100.101</name>
                        <metric>200</metric>
                    </qualified-next-hop>
                </route>
            </static>

Is this something you need or were you just testing? If it's the latter I'd recommend getting the image juniper/ffp-12.1X47-D20.7-packetmode. If you need this for production purposes let me know.

dbarrosop commented 5 years ago

Unfortunately I don't have the cycles to keep maintaining this project so as of now it's officially abandoned. If someone wants to fork the project and maintain it don't hesitate to contact me to add a link to the fork and direct existing and new users to the new fork. However, I'd recommend checking the following alternative:

yangify

Yangify has a cleaner interface that is easier to maintain and happens to be orders of magnitude faster and also more memory efficient.

If you are an existing user of napalm-yang and are worried about this announcement and/or want help migrating to some other project don't hesitate to contact me.

napalm-automation / napalm-yang

next-hop not being parsed in BGP Junos #143