Output can contain invalid start-ip/mask combinations

[kfeldmann]

Steps to reproduce:

1. Start with a sufficiently complicated list of cidrs
2. Pass the list through cidrmerge
3. Pass the output through cidrmerge again

In some cases, cidrmerge will detect bad cidrs in its own output.

Example:

$ cidrmerge < input.txt > output.txt
$ cidrmerge < output.txt 
ERROR: [52.70.0.0/14] Invalid starting address for /14. Try 52.68.0.0 or 52.72.0.0

[tuhaolam]

@kfeldmann Hi, i also met the issue what u said above, how can i fix the error? In addition, there is a other issue, this script can not totally merge all ip ranges to cidrs, and i have to merge the output once again and once again.

[tuhaolam]

like this: ERROR: [1.255.59.71/31] Invalid starting address for /31. Try 1.255.59.70 or 1.255.59.72

and the script following used to merge the output:

#!/bin/bash

#script for convert ip range 2 cidr.

[ $# -eq 0 ] && echo -e "$0 src_file" && exit 1
ip2cidr_py='cidrmerge.py'
src_ip=$1

#dst_ip1=$(python $ip2cidr_py < $src_ip)
python $ip2cidr_py < $src_ip > dst_ip1.txt || exit 1
#dst_ip2=$(python $ip2cidr_py < dst_ip1.txt)
python $ip2cidr_py < dst_ip1.txt > dst_ip2.txt || exit 1

while true
    do
        if [ `md5sum dst_ip1.txt| awk '{print $1}'` != `md5sum dst_ip2.txt| awk '{print $1}'` ];then
            src_ip="dst_ip2.txt"
            #dst_ip1=$(python $ip2cidr_py < $src_ip)
            python $ip2cidr_py < $src_ip > dst_ip1.txt
            #dst_ip2=$(python $ip2cidr_py < dst_ip1.txt)
            python $ip2cidr_py < dst_ip1.txt > dst_ip2.txt
        else
            echo "[[[ip2cidr results:]]]"
            cat dst_ip2.txt
            exit 0
        fi
    done

[kfeldmann]

Hi @tuhaolam,

Thank you for your feedback.

Unfortunately the algorithm that cidrmerge uses to convert its internal merged list of ip addresses back to a list of CIDRs for output is flawed and needs to be replaced. I have not yet taken the time to develop an accurate algorithm to use as a replacement. The cidrmerge program will not work correctly until that work is done.

As far as completely merging the inputs, please be aware that: Because of the way network masks work, CIDR ranges can start only at specific IPs. For example, not every list of 256 consecutive IP addresses can be made into a /24 CIDR. Depending on your inputs, it definitely may not be possible to merge everything. In some cases, your output list might be identical to your inputs (no merging possible at all). The cidrmerge program also does not support the capability to produce "approximate" CIDRs that loosely cover all input addresses, including any gaps and possibly some extra address space at the start or end. The goal is to only support exact merging.

Thank you

[kfeldmann]

I've finally fixed this issue. Also, as pointed out by @tuhaolam, there was another issue in that cidrmerge could fail to merge all possible combinations. I believe both of these issues are now resolved. Please let me know if you can still reproduce either of these issues, or if you find a new issue. Thank you.