Duplicate IPs

[consultoriaheitor]

Hello, I'm currently facing a dilemma. I provide consultancy services to various ISPs, and we have encountered a situation that we can't seem to resolve. I believe that the tool "topolograph" could be the solution to this problem. We need to identify which OSPF links have duplicate networks. For example, consider the following linear topology: router A connects to router B using the IP x.x.x.x/30, router B connects to router C using the IP y.y.y.y/30, and router C connects to router D using the IP z.z.z.z/30. The correct topology should be A<x.x.x.x/30>B<y.y.y.y/30>C<z.z.z.z/30>D.

Now, let's examine the first issue: A<x.x.x.x/30>B<y.y.y.y/30>C<x.x.x.x/30>D. How can we identify this duplication?

In the second problem, the topology is A<x.x.x.x/30>B<y.y.y.y/30>C<z.z.z.z/30>D<x.x.x.x/30>CLIENT, where the connection between router D and the CLIENT does not use OSPF. How can we address this issue?

[Vadims06]

Hi @consultoriaheitor, I hope that Topolograph will be helpful.

Regarding your request, I think it can quiet easy to solve. You need to upload LSDB to topolograph, then request all networks via API call

r_post = requests.post('https://topolograph.com/api/path/network', auth=('', ''))

It will return all networks in format below:

{
  "10.10.10.0/24": [
    {
      "cost": 1,
      "rid": "123.123.123.123"
    }
  ],
  "100.100.100.0/24": [
    {
      "cost": 1,
      "metric_type": 2,
      "rid": "123.123.123.123"
    },
  "x.x.x.x/30": [
    {
      "cost": 1,
      "rid": "A"
    },
    {
      "cost": 1,
      "rid": "B"
    },
    {
      "cost": 1,
      "rid": "C"
    },
    {
      "cost": 1,
      "rid": "C"
    }
  ]
}

So the x.x.x.x/30 should have four terminated routers. Filter all networks with a number of terminated routers more than two.

[consultoriaheitor]

Hello, your assistance has been extremely helpful. I've tested here and noticed that a simple request is not accepted; it requires specifying the network, which I don't know since that's what I'm trying to find out – if there are any duplicate networks in OSPF.

I made the following request:

curl -X 'POST' \ 'http://1.1.1.1:8080/api/path/network/' \ -H 'accept: application/json' \ -H 'Authorization: Basic XXXXXXXXXXXXXXX' \ -H 'Content-Type: application/json' \ -d '{"graph_time": "05Jan2024_21h19m35s_96_hosts"}'

And it returned the following:

{ "detail": "'src_ip_or_network' is a required property", "status": 400, "title": "Bad Request", "type": "about:blank" }

I tested on a local server because my account from the site https://topolograph.com/ was created via Google, and I couldn't create a password to use it.

[Vadims06]

Hi @consultoriaheitor , sorry, my bad, it's not a POST request, it's a GET.

>>> r_get = requests.get(f'http://{TOPOLOGRAPH_HOST}:{TOPOLOGRAPH_PORT}/api/network/{graph_time}', auth=(TOPOLOGRAPH_WEB_API_USERNAME_EMAIL, TOPOLOGRAPH_WEB_API_PASSWORD))
>>> len(r_get.json())
32

[consultoriaheitor]

Hello,

I have successfully gathered the information I needed, and the use of Topolograph was extremely helpful. Thank you for your assistance.

However, I am still interested in integrating this functionality into a web-based application, which would be more convenient. Attached is the Python script I developed, and I would appreciate it if you could evaluate the feasibility of incorporating this feature.

If you have any suggestions for better API usage or programming logic, I am open to recommendations. As a background, I am in the Telecom sector and not deeply versed in programming. I occasionally dabble in scripting to streamline my tasks.

import requests
import argparse
import itertools

# Set up argument parser
parser = argparse.ArgumentParser(description='Analyze Network Blocks with Various Rules')
parser.add_argument('-U', '--url', type=str, required=True, help='Base URL up to /api, e.g., https://topolograph.com')
parser.add_argument('-u', '--user', type=str, required=True, help='API Username')
parser.add_argument('-p', '--password', type=str, required=True, help='API Password')
parser.add_argument('-g', '--graph_time', type=str, required=True, help='Graph Time Parameter')
parser.add_argument('-v', '--verbose', action='store_true', help='Enable verbose output')
parser.add_argument('-e', '--exceptions', type=str, help='Custom exception rules as CIDR:COUNT, e.g., 30:2,32:1')
parser.add_argument('-n', '--neighbor', action='store_true', help='Enable neighbor analysis')

# Parse arguments
args = parser.parse_args()

# Function to parse custom exception rules
def parse_exception_rules(rules_str):
    rules = {}
    if rules_str:
        for rule in rules_str.split(','):
            cidr, count = rule.split(':')
            rules[cidr] = int(count)
    return rules

# Function to check if all RIDs in a network block are neighbors
def are_all_rids_neighbors(neighbor_data, rids):
    for rid1, rid2 in itertools.combinations(rids, 2):
        if not (is_direct_neighbor(neighbor_data, rid1, rid2) or is_direct_neighbor(neighbor_data, rid2, rid1)):
            return False
    return True

def is_direct_neighbor(neighbor_data, rid1, rid2):
    return any(neighbor['src'] == rid1 and neighbor['dst'] == rid2 for neighbor in neighbor_data)

# Function to make a GET request with basic authentication
def make_auth_request(url, user, password):
    response = requests.get(url, auth=(user, password))
    if response.status_code != 200:
        print(f"Request failed: {response.status_code}")
        return None
    return response.json()

# Function to get neighbor data
def get_neighbor_data(url, user, password, graph_time):
    neighbor_url = f'{url}/api/diagram/{graph_time}/edges'
    return make_auth_request(neighbor_url, user, password)

# Function to analyze network blocks with various rules
def analyze_network_blocks(network_data, custom_rules, neighbor_data):
    network_info = {}
    duplicates = []

    for network, details in network_data.items():
        cidr = network.split('/')[-1]
        if network not in network_info:
            network_info[network] = {'rid_count': 0, 'rids': []}
        for detail in details:
            if detail['rid'] not in network_info[network]['rids']:
                network_info[network]['rids'].append(detail['rid'])
                network_info[network]['rid_count'] += 1

        rid_count = network_info[network]['rid_count']
        if args.exceptions:
            # Custom rules logic
            max_count = custom_rules.get(cidr, 1)
            if rid_count > max_count:
                duplicates.append((network, rid_count, network_info[network]['rids']))
        elif args.neighbor:
            # Neighbor analysis logic
            if cidr != "32" and rid_count > 1 and not are_all_rids_neighbors(neighbor_data, network_info[network]['rids']):
                duplicates.append((network, rid_count, network_info[network]['rids']))
            elif cidr == "32" and rid_count > 1:
                duplicates.append((network, rid_count, network_info[network]['rids']))
        else:
            # Default rules logic
            if (cidr == "32" and rid_count > 1) or (cidr == "30" and rid_count > 2) or (cidr not in ["32", "30"] and rid_count > 1):
                duplicates.append((network, rid_count, network_info[network]['rids']))

        if args.verbose:
            print(f"Network block: {network}, RID Count: {rid_count}, RIDs: {network_info[network]['rids']}")

    if duplicates:
        print("\nDuplicate Networks:")
        for network, count, rids in duplicates:
            print(f"Network block: {network}, Duplicate Count: {count}, RIDs: {rids}")
    else:
        print("\nNo duplicate networks found.")

# Function to check if RIDs are neighbors
def is_neighbor(neighbor_data, rids):
    if neighbor_data is not None:
        for neighbor in neighbor_data:
            if neighbor['src'] in rids and neighbor['dst'] in rids:
                return True
    return False

# Construct the full URL for fetching network data
network_url = f'{args.url}/api/network/{args.graph_time}'

print("Starting network analysis script...")

# Parse custom exception rules
custom_rules = parse_exception_rules(args.exceptions)

# Fetch network data
network_data = make_auth_request(network_url, args.user, args.password)

# Fetch neighbor data if needed
neighbor_data = None
if args.neighbor:
    neighbor_data = get_neighbor_data(args.url, args.user, args.password, args.graph_time)

if network_data:
    print("Network data fetched successfully.")
    analyze_network_blocks(network_data, custom_rules, neighbor_data)
else:
    print("Failed to fetch network data.")

[Vadims06]

Thanks a lot for sharing your code! I'm glad that Topolograph was helpful! It's possible to include it on the web page. What format of output data will be in that case? Sample of results ( networks with more than two terminated nodes )	Network	Number of terminated nodes	Node names
10.0.0.0/24	4	172.16.1.2, 172.26.1.2, 172.30.2.1, 178.20.3.1

[consultoriaheitor]

Yes, that already seems interesting, but it would be beneficial to identify if any of these IPs are neighbors using a symbol such as <->. In the current code without additional parameters, the verification is as follows: for /32, only one is possible; for /30, a maximum of two; and for other CIDRs, only one. When the -n option is used, the code checks for neighbors and analyzes, for instance, if a /30 has routers A and B as neighbors, as well as routers C and D, which should not be the case. Additionally, the -e option allows me to specify the maximum number of each CIDR, like -e 31:2 for a /31 which can have a maximum of two.

Regarding the display, I am unsure about the most user-friendly way to visualize the issues based on the specifications I choose, such as whether to analyze neighbors or not and custom values for considering duplicates. However, the proposed method of showing the quantity and using a symbol like "IPa <-> IPb, IPc <-> IPd, IPe" seems like it would also be effective.

[Vadims06]

@consultoriaheitor added the button under Analytics, please check.

[consultoriaheitor]

Hello,

Firstly, thank you very much for implementing this function. I tested it and found that it does not work in the Edge browser, although it does work in Chrome. However, I encountered an issue where, if I click on 'Show' and then try to minimize it for analysis, the display disappears and I have to click on 'Analyze' again, which is somewhat inconvenient. Another suggestion is to make the display more compact for easier analysis and, if possible, add an additional column. If any of the hosts are neighbors, perhaps you could indicate this by changing the color of the row or adding a marker, as neighbor duplication is a critical issue in MPLS.

Additionally, it would be useful to have a feature where hovering the mouse over a host highlights it on the map, possibly by enlarging the point. Clicking on a host could focus on it by zooming in. This would greatly enhance the user experience in identifying and analyzing network relationships.

[consultoriaheitor]

I apologize, the Edge browser is actually working fine, it was a cache issue.

[Vadims06]

@consultoriaheitor , thank you for your feedback! Could you please give more details about If any of the hosts are neighbors, perhaps you could indicate this by changing the color of the row or adding a marker, as neighbor duplication is a critical issue in MPLS. so if we have four hosts, which have the same subnet and if they are neighbors - color it as green. What is the best way to mark the case, when there are two pairs of two nodes, which are neighbors?..

Clicking on a host could focus on it by zooming in.

Currently, clicking on a host pops up new window with all subnets terminated on o node. Zooming is available via spinning the wheel on a mouse. Did you find it useful?

[Vadims06]

I split neighbor list to neighbor groups. So result will be like this	Network	Number of terminated nodes	Node names
10.0.0.0/24	4	[172.16.1.2, 172.26.1.2], [172.30.2.1, 178.20.3.1]

[Vadims06]

Hi @consultoriaheitor , I'm closing this issue since most analysis has been implemented. I would be grateful to learn more about neighbor duplication is a critical issue in MPLS. If you have time, please describe here or in the email. Thank you!

[consultoriaheitor]

This is already excellent in the way it is represented. Regarding the feature of clicking on an IP and highlighting it on the map, I would like to have the ability to click on an IP in the list and have the map focus on that IP.

[consultoriaheitor]

The issue with MPLS being affected by IP duplications in the network stems from the fact that labels are generated based on IP blocks, which can be duplicated. As a result, the same IP block may be associated with different labels. This leads to a situation where the same block, with different labels, is propagated to other routers, including the one with the duplicated IP. This becomes problematic when implementing MPLS Traffic Engineering (MPLS-TE) or Resource Reservation Protocol Traffic Engineering (RSVP-TE), as it can degrade performance and lead to routing bugs in relation to MPLS-TE, among other issues. Therefore, the issue of block duplication across links is critical in MPLS environments, primarily due to the generation of duplicate labels for the same block. In routers, this causes confusion when recalculating the path for TE or RSVP-TE.