ljyfree commented 4 years ago

Hi Tnaganawa

As you suggestion, I can move on but was blocked by another issue. Step 1: add qfx physical router Step 2: add BGP router as_number

[root@master01 ~]# tailf /var/log/contrail/contrail-device-manager.log 
04/19/2020 10:27:56 PM [contrail-device-manager] [WARNING]: It is not recommended to use 'get' or 'multiget' pycassa methods. It's better to use 'xget' or 'get_range' methods due to thrift limitations
04/19/2020 10:27:56 PM [contrail-device-manager] [ERROR]: Router 192.168.122.103: error: as_number: 'b'64512'': Use format 'x' or 'x.y' to specify number 'x' (Range 1 .. 4294967295) or number with higher 2 bytes 'x' (Range 0 .. 65535) ay
error: syntax error, expecting <rpc> or </rpc>
04/19/2020 10:27:57 PM [contrail-device-manager] [ERROR]: Router 192.168.122.103: error: as_number: 'b'64512'': Use format 'x' or 'x.y' to specify number 'x' (Range 1 .. 4294967295) or number with higher 2 bytes 'x' (Range 0 .. 65535) ay
error: syntax error, expecting <rpc> or </rpc>
04/19/2020 10:27:57 PM [contrail-device-manager] [WARNING]: It is not recommended to use 'get' or 'multiget' pycassa methods. It's better to use 'xget' or 'get_range' methods due to thrift limitations
04/19/2020 10:27:57 PM [contrail-device-manager] [ERROR]: Router 192.168.122.103: error: as_number: 'b'64512'': Use format 'x' or 'x.y' to specify number 'x' (Range 1 .. 4294967295) or number with higher 2 bytes 'x' (Range 0 .. 65535) ay
error: syntax error, expecting <rpc> or </rpc>
04/19/2020 10:29:57 PM [contrail-device-manager] [ERROR]: Router 192.168.122.103: error: as_number: 'b'64512'': Use format 'x' or 'x.y' to specify number 'x' (Range 1 .. 4294967295) or number with higher 2 bytes 'x' (Range 0 .. 65535) ay
error: syntax error, expecting <rpc> or </rpc>
^C
[root@master01 ~]#

As you see, as_number was set to "b'64512'" instead of "64512". And I could not find out where the error was reported according to the message. Same issue when I change from "qfx5100" to "mx".

Any suggestion? Thank you anyway.

ljyfree commented 4 years ago

Print the config that TF try to download

    <configuration>
        <groups operation="replace">
            <junos:comment>/* Contrail Generated Group Config */</junos:comment>
            <name>__contrail__</name>
            <routing-options>
                <junos:comment>/* Global Routing Options */</junos:comment>
                <router-id>3.3.3.3</router-id>
                <route-distinguisher-id>3.3.3.3</route-distinguisher-id>
                <autonomous-system>b'64512'</autonomous-system>
                <resolution>
                    <rib>
                        <name>bgp.rtarget.0</name>
                        <resolution-ribs>inet.0</resolution-ribs>
                    </rib>
                </resolution>
            </routing-options>
            <protocols>
                <junos:comment>/* Protocols Configuration */</junos:comment>
                <bgp>
                    <group>
                        <junos:comment>/* overlay_bgp: BGP Router: qfx5100-48t-1, UUID: f0afa9d3-d388-45ea-b6e2-c5319e9664e0 */</junos:comment>
                        <name>b'_contrail_asn-64512'</name>
                        <type>internal</type>
                        <local-address>3.3.3.3</local-address>
                        <hold-time>90</hold-time>
                        <family>
                            <evpn>
                                <signaling></signaling>
                            </evpn>
                            <route-target></route-target>
                        </family>
                        <neighbor>
                            <junos:comment>/* overlay_bgp: BGP Router: master01.local, UUID: b04bc39c-57ea-4f0e-b04a-3d2737ce242f */</junos:comment>
                            <name>192.168.122.177</name>
                            <peer-as>64512</peer-as>
                        </neighbor>
                    </group>
                </bgp>
            </protocols>
            <policy-options>
                <junos:comment>/* Policy Options */</junos:comment>
                <community>
                    <name>_contrail_switch_policy_</name>
                    <members>b'target:64512:1'</members>
                </community>
            </policy-options>
            <switch-options>
                <vtep-source-interface>lo0.0</vtep-source-interface>
            </switch-options>
        </groups>
        <apply-groups>
            <name>__contrail__</name>
        </apply-groups>
    </configuration>
</config>

There are three "b'xx'" in config_str as followes:

<autonomous-system>b'64512'</autonomous-system>
 <name>b'_contrail_asn-64512'</name>
<members>b'target:64512:1'</members>

tnaganawa commented 4 years ago

Hi, Johnny,

Yeah, I noticed this issue recently, but I don't have complete fix yet .. :(

adding future.utils.native_str to each string would mitigate the issue for each parameter https://github.com/Juniper/contrail-controller/blob/master/src/config/device-manager/device_manager/plugins/juniper/juniper_conf.py#L356

If possible, could you change config-device-manager's version to opencontrailnightly/1910-latest? /etc/contrail/config/docker-compose.yaml have this info.

docker-compose -f /etc/contrail/config/docker-compose.yaml down vi /etc/contrail/config/docker-compose.yaml ## change device-manager's container tag docker-compose -f /etc/contrail/config/docker-compose.yaml up -d

ljyfree commented 4 years ago

Hi Tnaganawa

Thanks for you guide and I will try tomorrow.
A quick question : How could I identify the version of the docker image? The tag is not reliable ，which could be set manually.

ljyfree commented 4 years ago

Hi Tnaganawa

It works as you suggested that apply-groups __contrail__ was deployed successfully.
All I could say is--thanks.

ljyfree commented 4 years ago

I found the isuse "b'64512'" still exist on R2003 docker image, with more configure with "b''"

            <autonomous-system>b'64512'</autonomous-system>
            <dynamic-tunnels>
                <dynamic-tunnel>
                    <name>b'_contrail_asn-64512'</name>
                    <source-address>2.2.2.2</source-address>
                    <gre></gre>
                    <destination-networks>
                        <junos:comment>/* IP Fabric Subnet */</junos:comment>
                        <name>b'192.168.122.0/24'</name>
                    </destination-networks>

tnaganawa commented 4 years ago

yeah, I actually have a fix for this, but it is not merged so far .. :(

native_str statement needs to be added in several places

If community build began, perhaps I could upload this again. https://wiki.tungsten.io/display/TUN/%5BProposal%5D+Release+Process

ljyfree commented 4 years ago

Hi tnaganawa ,

Could you please give some guidance so I can fix them locally and compile ,without waiting for the community build?

ljyfree commented 4 years ago

I am really confused by "export_xml" , since I could not find where this function was defined.

# in juniper.py

    def serialize(self, config):
        xml_data = StringIO()
        config.export_xml(xml_data, 1)
        xml_str = xml_data.getvalue()
        return xml_str.replace("comment>", "junos:comment>", -1)
    # end serialize

tnaganawa commented 4 years ago

Hi, Could you try this patch?

Some python 3 byte string needs to be forced to be python2 string with native_str function.

I believe mx.conf also needs to be modified, but I didn't have complete fix now..


[root@centos211 device_manager]# diff -u juniper_conf.py.xxx juniper_conf.py
--- juniper_conf.py.xxx 2020-03-31 07:39:33.000000000 +0900
+++ juniper_conf.py     2020-04-09 16:54:12.415587923 +0900
@@ -8,6 +8,7 @@

from future import standard_library
standard_library.install_aliases()
+from future.utils import native_str
from builtins import str
from builtins import object
from ncclient import manager
@@ -349,13 +350,13 @@

 def add_dynamic_tunnels(self, tunnel_source_ip,
                          ip_fabric_nets, bgp_router_ips):
-        dynamic_tunnel = DynamicTunnel(name=DMUtils.dynamic_tunnel_name(self.get_asn()),
+        dynamic_tunnel = DynamicTunnel(name=native_str(DMUtils.dynamic_tunnel_name(self.get_asn())),
                                    source_address=tunnel_source_ip, gre='')
     if ip_fabric_nets is not None:
         for subnet in ip_fabric_nets.get("subnet", []):
             dest_net = subnet['ip_prefix'] + '/' + str(subnet['ip_prefix_len'])
             dynamic_tunnel.add_destination_networks(
-                    DestinationNetworks(name=dest_net,
+                    DestinationNetworks(name=native_str(dest_net),
                                     comment=DMUtils.ip_fabric_subnet_comment()))

     for r_name, bgp_router_ip in list(bgp_router_ips.items()):
@@ -465,7 +466,7 @@
         bgp_group.set_type('external')
         bgp_group.set_multihop('')
     else:
-            bgp_group.set_name(DMUtils.make_bgp_group_name(self.get_asn(), False))
+            bgp_group.set_name(native_str(DMUtils.make_bgp_group_name(self.get_asn(), False)))
         bgp_group.set_type('internal')
         self.add_ibgp_export_policy(self.bgp_params, bgp_group)
     bgp_group.set_local_address(self.bgp_params['address'])
@@ -519,7 +520,7 @@
     if self.global_routing_options_config is None:
         self.global_routing_options_config = RoutingOptions(comment=DMUtils.routing_options_comment())
     self.global_routing_options_config.set_route_distinguisher_id(self.bgp_params['identifier'])
-        self.global_routing_options_config.set_autonomous_system(str(self.get_asn()))
+        self.global_routing_options_config.set_autonomous_system(native_str(self.get_asn()))
 # end set_as_config

 def set_bgp_group_config(self):
[root@centos211 device_manager]#
[root@centos211 device_manager]#
[root@centos211 device_manager]# diff -u qfx_conf.py.xxx qfx_conf.py
--- qfx_conf.py.xxx     2020-03-31 07:39:33.000000000 +0900
+++ qfx_conf.py 2020-04-09 17:16:37.733529116 +0900
@@ -7,6 +7,7 @@
configuration manager
"""

+from future.utils import native_str from builtins import str from db import * from dm_utils import DMUtils @@ -438,7 +439,7 @@

no vn config then no need to configure vrf target

         return
     self.global_switch_options_config.add_vrf_target(VniTarget(auto=''))

switch_options_community = DMUtils.get_switch_vrf_import(self.get_asn())
switch_options_community = native_str(DMUtils.get_switch_vrf_import(self.get_asn())) self.global_switch_options_config.add_vrf_target(VniTarget(community=switch_options_community)) self.set_global_export_policy()
end init_global_switch_opts

@@ -490,7 +491,7 @@ self.policy_config.add_community(export_comm)

add community for switch options
```
 comm = CommunityType(name=DMUtils.get_switch_policy_name())
```
comm.add_members(DMUtils.get_switch_vrf_import(self.get_asn()))
comm.add_members(native_str(DMUtils.get_switch_vrf_import(self.get_asn()))) self.policy_config.add_community(comm)
end set_route_targets_config

[root@centos211 device_manager]#

ljyfree commented 4 years ago

Hi tnaganawa

It works for R2003 and thank you sincerely for your time again.

tnaganawa / tungstenfabric-docs

TF could not download BGP configuration since the format error for as_number #2

no vn config then no need to configure vrf target

end init_global_switch_opts

add community for switch options

end set_route_targets_config