IPv6 Parsing

[JakeYare]

Just want to start by saying the Blacklist Management project is awesome, and you're awesome for maintaining it! Just wanted to report an issue using the default lists FW-Blacklist-URLs, it seems parsing IPv6 is broken at this time. I had to remove each of the IPv6 lists, including BlockList.de which had a few IPv6 addresses, in order for updBlackList.sh to succeed. After removing each list which contained IPv6 addresses, I was sucessfully able to import the IPSet: IPv4 blocklist items fetched: 23470, unique: 21522, final: 21522 Total IPv4 prefix length count (including hosts): 27 IPv6 blocklist items fetched: 0, unique: 0, final: 0 Total IPv6 prefix length count (including hosts): 0

Currently on EdgeRouter X v2.0.8 but also tried on 1.10.10

ubnt@ubnt:~$ sudo /config/scripts/updBlackList.sh Missing executable '/usr/bin/iprange'. Will not optimize IPsets Starting at 00:52:56 MST Wed 11 Dec 2019 Fetching 'https://www.team-cymru.org/Services/Bogons/fullbogons-ipv4.txt' Fetching 'https://www.team-cymru.org/Services/Bogons/fullbogons-ipv6.txt' Fetching 'https://rules.emergingthreats.net/fwrules/emerging-Block-IPs.txt' Fetching 'https://www.spamhaus.org/drop/drop.txt' Fetching 'https://www.spamhaus.org/drop/dropv6.txt' Fetching 'https://www.spamhaus.org/drop/edrop.txt' Fetching 'https://lists.blocklist.de/lists/all.txt' Fetching 'https://iplists.firehol.org/files/firehol_level1.netset' Fetching 'https://www.okean.com/chinacidr.txt' Processing block file list (IPv4): ' 00_www.team-cymru.org_fullbogons-ipv4.txt 01_www.team-cymru.org_fullbogons-ipv6.txt 02_rules.emergingthreats.net_emerging-Block-IPs.txt 03_www.spamhaus.org_drop.txt 04_www.spamhaus.org_dropv6.txt 05_www.spamhaus.org_edrop.txt 06_lists.blocklist.de_all.txt 07_iplists.firehol.org_firehol_level1.netset 08_www.okean.com_chinacidr.txt' Applying IPset (IPv4) Processing block file list (IPv6): ' 00_www.team-cymru.org_fullbogons-ipv4.txt 01_www.team-cymru.org_fullbogons-ipv6.txt 02_rules.emergingthreats.net_emerging-Block-IPs.txt 03_www.spamhaus.org_drop.txt 04_www.spamhaus.org_dropv6.txt 05_www.spamhaus.org_edrop.txt 06_lists.blocklist.de_all.txt 07_iplists.firehol.org_firehol_level1.netset 08_www.okean.com_chinacidr.txt' Applying IPset (IPv6) ipset v6.30: Error in line 2: Syntax error: cannot parse 1000::: resolving to IPv4 address failed FATAL: inet6 ipset restore failed: error 1

[AndreasHD23]

I have the same issue with EdgeRouter X SFP v2.0.9-hotfix.2 Any news on this? Applying IPset (IPv6) ipset v6.30: Error in line 2: Syntax error: cannot parse 1000::: resolving to IPv4 address failed FATAL: inet6 ipset restore failed: error 1

[WaterByWind]

This likely is due to content in the lists so would need at least the list of those configured to try to replicate and find the bad text. Or a copy of the lists (cached in /tmp/.BL) provided from the time when this happens.

[AndreasHD23]

Yes it's content in one of the lists. From /tmp/.BL/03_www.spamhaus.org_dropv6.txt 2602:ffa0::/36 ; SBL262027 <-- last one to parse fine 2401:c580::/32 ; SBL246818 <-- throws error

Error always is ipset v6.30: Error in line 2: Syntax error: cannot parse 2401:c580::: resolving to IPv4 address failed

Just like JakeYare in 2019 I am currently not using iprange. Would that make a difference? Considering there still seems to be no IPv6 support in iprange.

This is the debug output after the last curl download of the last list:

Updated block file list: ' 00_rules.emergingthreats.net_emerging-Block-IPs.txt 01_www.spamhaus.org_drop.txt 02_www.spamhaus.org_dropv6.txt 03_www.spamhaus.org_edrop.txt 04_lists.blocklist.de_all.txt 05_iplists.firehol.org_firehol_level1.netset'
doProcess4()
Processing block file list (IPv4): ' 00_rules.emergingthreats.net_emerging-Block-IPs.txt 01_www.spamhaus.org_drop.txt 02_www.spamhaus.org_dropv6.txt 03_www.spamhaus.org_edrop.txt 04_lists.blocklist.de_all.txt 05_iplists.firehol.org_firehol_level1.netset'
Counting total IPv4 addresses received
Counting unique IPv4 addresses received
Counting IPv4 filtered addresses
Counting IPv4 address prefixes
doUpdate4()
Applying IPset (IPv4)
Creating temp inet IPset
Swapping temp and ThreatBlacklist4 IPsets
Destroying temp inet IPset
doProcess6()
Processing block file list (IPv6): ' 00_rules.emergingthreats.net_emerging-Block-IPs.txt 01_www.spamhaus.org_drop.txt 02_www.spamhaus.org_dropv6.txt 03_www.spamhaus.org_edrop.txt 04_lists.blocklist.de_all.txt 05_iplists.firehol.org_firehol_level1.netset'
Counting total IPv6 addresses received
Counting unique IPv6 addresses received
Counting IPv6 filtered addresses
Counting IPv6 address prefixes
doUpdate6()
Applying IPset (IPv6)
Creating temp inet6 IPset
ipset v6.30: Error in line 2: Syntax error: cannot parse 2001:41d0:1:e906::1: resolving to IPv4 address failed
die()
FATAL: inet6 ipset restore failed: error 1
atExit()
doSendEmail()

[AndreasHD23]

Ok, nevermind... SOLUTION: Be careful to create an ipv6-network-group during initial setup. With just a network-group it'll have have an ipset headet with family inet... which causes ipv6 addresses to fail. ipset needs an family inet6 header in order to process ipv6 - this gets created when using ipv6-network-group. Thanks for your time!