Feature request for URL filter

[GoogleCodeExporter]

Currently URL filter only supports 128 items.
Could we have increased items, say 256 items?

I know there are limitation with the RAM or something.
But could there be a solution, or alternative (without the need to purchase 
other router)?

Original issue reported on code.google.com by wind77 on 25 Nov 2013 at 3:39

[GoogleCodeExporter]

URL filter is very-very slow! Performance degradation is catastrophic when use 
more items.

Use ipset.

Original comment by andy.pad...@gmail.com on 25 Nov 2013 at 3:42

[GoogleCodeExporter]

Thanks for reply.
Could you provide some guidance on ipset?

Original comment by wind77 on 25 Nov 2013 at 3:45

[GoogleCodeExporter]

http://ipset.netfilter.org/ipset.man.html

Original comment by Dr.Sydorenko.O on 25 Nov 2013 at 6:36

[GoogleCodeExporter]

I wrote a script for the N56U which automates the IPSet banning/unbanning 
process along with a few other features feel free to use it. You will be able 
to ban millions of IP's without any noticeable performance degradation.

http://pastebin.com/nSYB1ErS

Copy the contents from the link above to the following file via SSH "nano 
/opt/bin/firewall" then save the file.

After doing so you will need to chmod the file using the following command 
"chmod +x /opt/bin/firewall"

You will also need to copy the contents of the following link to a file in the 
admin GUI.

http://pastebin.com/ZiVh9hAp

Administration - Tweaks - "Run after Router started:"

Original comment by c_u_late...@hotmail.com on 26 Nov 2013 at 5:45

[GoogleCodeExporter]

Running the script is easy after doing the steps above, just type the word 
"firewall xxxx" in SSH replacing "xxxx" with one of the following commands.

##############################
"unban"          # <-- Remove Single IP From Blacklist
"unbanall"           # <-- Unbans All IPs In Blacklist
"removeall"          # <-- Remove All Entries From Blacklist
"save"                   # <-- Save Blacklists to /opt/tmp/ipset.txt
"ban"                    # <-- Adds Entry To Blacklist
"country"                # <-- Adds entire country to blacklist
"bancountry"             # <-- Bans specified countries in this file
"hideme"         # <-- Switch to unrestricted DNS (proxydns.co)
"backup"         # <-- Backup IPSet Rules to /opt/tmp/ipset2.txt
##############################

Original comment by c_u_late...@hotmail.com on 26 Nov 2013 at 5:48

[GoogleCodeExporter]

Very useful script, GJ sir.

/Kitch

Original comment by kitch2400 on 26 Nov 2013 at 2:48

[GoogleCodeExporter]

No problem, hopefully in the future similar functionality will be added by 
default (Or a wiki guide with the script above so people are more aware of 
IPSets usefulness)

Original comment by c_u_late...@hotmail.com on 27 Nov 2013 at 3:42

[GoogleCodeExporter]

Many thanks to all the experts who helped!
I'm no a linux expert, so I tried to google and find out about how to insert 
the script file.
So first thing I realize is that: my opt folder is empty.
I assume the firmware needs to be updated before 1st use, so i entered "opkg 
update" in putty.
Then I was prompted "not found". 
I checked through user guide, and my understanding now is that: i need a USB 
disk to install Optware, and the USB disk must remained inserted to router in 
order to use Optware.
Am I right thus far?
Thanks in advance!

Original comment by wind77 on 27 Nov 2013 at 9:10

[GoogleCodeExporter]

Yes you will need to install entware for full functionality on a small usb/hdd. 
There is a guide on the Wiki located here that should answer all your questions.

Original comment by c_u_late...@hotmail.com on 28 Nov 2013 at 1:49

[GoogleCodeExporter]

https://code.google.com/p/rt-n56u/wiki/HowToConfigureEntware

Original comment by c_u_late...@hotmail.com on 28 Nov 2013 at 1:51

[GoogleCodeExporter]

Also on the note of IPSet, v6.20.1 is now available with loads of changes.

6.20.1
Kernel part changes
netfilter: ipset: remove duplicate define (Michael Opdenacker)
net->user_ns is available starting from 3.8, add compatibility checking 
(reported by Jan Engelhardt)
Fix memory allocation for bitmap:port (reported by Quentin Armitage)
Avoid clashing with configured kernel in [CONFIG_]IP_SET_MAX
The unnamed union initialization may lead to compilation error (reported by 
Husnu Demir)
Use dev_net() instead of the direct access to ->nd_net (reported by the kbuild 
test robot)
Userspace changes
build: fix incorrect library versioning (Jan Engelhardt)
netfilter: ipset: Fix configure failure when --with-kmod=no (Oliver Smith)
Avoid clashing with configured kernel in [CONFIG_]IP_SET_MAX
6.20
Kernel part changes
Compatibility code is modified not to rely on kernel version numbers
Use netlink callback dump args only
Add hash:net,port,net module to kernel (Oliver Smith)
Add net namespace for ipset (Vitaly Lavrov)
Use a common function at listing the extensions of the elements
For set:list types, replaced elements must be zeroed out
Fix hash resizing with comments
Support comments in the list-type ipset (Oliver Smith)
Support comments in bitmap-type ipsets (Oliver Smith)
Support comments in hash-type ipsets (Oliver Smith)
Support comments for ipset entries in the core (Oliver Smith)
Add hash:net,net module to kernel (Oliver Smith)
Fix serious failure in CIDR tracking (Oliver Smith)
list:set: make sure all elements are checked by the gc
Support extensions which need a per data destroy function
Generalize extensions support
Move extension data to set structure
Rename extension offset ids to extension ids
Prepare ipset to support multiple networks for hash types
Introduce new operation to get both setname and family
Validate the set family and not the set type family at swapping (Bug reported 
by Quentin Armitage, netfilter bugzilla id #843)
Consistent userspace testing with nomatch flag
Skip really non-first fragments for IPv6 when getting port/protocol
ipset standalone package needs to ship em_ipset.c (reported by Jan Engelhardt)
Userspace changes
Missing comment support added to hash:ip,port,ip and hash:net,iface types
Compatibility code is modified not to rely on kernel version numbers
Add userspace code to support hash:net,port,net kernel module (Oliver Smith)
Tests added to check comment extension
Add new userspace set revisions for comment support (Oliver Smith)
Support comments in the userspace library (Oliver Smith)
Rework the "fake" argument parsing for ipset restore (Oliver Smith)
Add userspace code to support hash:net,net kernel module (Oliver Smith)
Add test to verify CIDR tracking
configure: uclinux is also linux (Gustavo Zacarias)
Add specifying protocol for bitmap:port (Quentin Armitage)
Remove artifical restriction of netmask values for hash:ip type (Reported by 
Quentin Armitage, netfilter bugzilla id #844)
Make sure called test scripts can be executed (reported by Tomas Budai)
Manpage fix: not just identical, but compatible type of sets can be swapped 
(Reported by Quentin Armitage, netfilter bugzilla id #843)
Fix error message typo (Reported by Quentin Armitage, netfilter bugzilla id 
#843)
Parse option "family" first, because other options may depend on it (Bug 
reported by Quentin Armitage, closed netfilter bugzilla #841)
Change 2nd parameter type of ipset_parse_elem (Quentin Armitage)
Report broken netlink messages in debug mode
Fix hyphen used as minus sign in manpage (Neutron Soutmun)
libipset.pc must be installed via 'make install' (Eric Leblond)

Original comment by c_u_late...@hotmail.com on 29 Nov 2013 at 1:49

[GoogleCodeExporter]

If you have any questions/issues about Entware software repository, you must 
search/ask its at http://code.google.com/p/wl500g-repo

Original comment by Dr.Sydorenko.O on 29 Nov 2013 at 3:35

[GoogleCodeExporter]

IPSet is built into the firmware just like IPTables.. :P

Original comment by c_u_late...@hotmail.com on 29 Nov 2013 at 3:43