search

ScriptTiger / Hosts-BL

Simple tool to handle hosts file black lists that can remove comments, remove duplicates, compress to 9 domains per line, add IPv6 entries, as well as can convert black lists to multiple other black list formats compatible with other software.
https://scripttiger.github.io/more/
MIT License
53 stars 4 forks source link

Please can you tell how to use it on windows? #6

Closed SaumayTyagi closed 1 month ago

SaumayTyagi commented 1 month ago

Please can you tell how to use it on windows . i am not tech savvy but want to compress the hosts file . what to do . please give little detailed steps.

ScriptTiger commented 1 month ago

If you're converting a hosts file from Steven Black, simply left-click and hold on the hosts file you want to convert, drag it over the hosts-bl executable, and drop it. Several moments later, a new compressed hosts file will appear.

If you're converting a hosts file that is not from Steven Black and it is unreasonably large, such as 100+ mb in size, you may want to increase the hash size to reduce the possibility of collisions.

For further details, please refer to the README: https://github.com/ScriptTiger/Hosts-BL

Windows release downloads:

64-bit Windows: https://github.com/ScriptTiger/Hosts-BL/releases/download/11SEP2024/hosts-bl_windows_amd64.zip

32-bit Windows: https://github.com/ScriptTiger/Hosts-BL/releases/download/11SEP2024/hosts-bl_windows_386.zip

SaumayTyagi commented 1 month ago

sorry , but what you mean by increase hash size , please can you tell?

SaumayTyagi commented 1 month ago

And what is meant by collisions .

ScriptTiger commented 1 month ago

Because large hosts files cannot fit into memory, every domain name is mapped to hashes and operations are performed on the hashes instead of the domain names, so acting on the domain names by proxy. The size of those hashes can be from 64 bits to 256 bits. With 64 bits, that's a total of 18,446,744,073,709,551,615 possible hashes. With 256 bits, that's a total of 115,792,089,237,316,195,423,570,985,008,687,907,853,269,984,665,640,564,039,457,584,007,913,129,639,935 hashes. With a 64-bit hash, there is roughly a 1 in 4 billion chance of a collision occurring, varying by what hashing algorithm is used. If a collision occurs in hosts-bl, one of the colliding domains will get dropped off the list. Even though the probability of that causing a problem is extremely low, hosts-bl is still a security tool at its core and thus accounts for this by allowing you to increase the hash size to be more resistant to such collisions occurring.

For high security use cases where collisions need to be reduced 100%, I have also intentionally kept the legacy version available for download, which does not hash at all and directly compares strings, but it breaks with larger hosts files due to memory constraints.

Legacy Windows release downloads:

64-bit Windows: https://github.com/ScriptTiger/Hosts-BL/releases/download/13APR2023/hosts-bl_windows_amd64.zip

32-bit Windows: https://github.com/ScriptTiger/Hosts-BL/releases/download/13APR2023/hosts-bl_windows_386.zip

SaumayTyagi commented 1 month ago

Thank you very much . It worked with my 15MB hosts file . though i still dont know what hash and collision is?

ScriptTiger commented 1 month ago

In this particular case, hashes are fixed-length unsigned integers representing arbitrary strings, the domain names more specifically. Hashes have long been used in security applications, such as passwords, ATM pin numbers, etc. However, since hashes are fixed length but strings are variable length, they are obviously not a direct 1-to-1 representation. This means that when generating a hash for 2 different strings, there is a small chance that both strings could result in the same hash, which would be a collision. While this chance is extremely small, some security applications have different acceptable thresholds and require hashes of larger sizes to reduce that chance even further.

In your case, as your hosts file is only 15 mb, hashes should be of no concern to you, as I clearly stated in my first comment "100+ mb" as just a very rough example of a large hosts file. Your hosts file is not large and the defaults should be perfectly acceptable in your case.

SaumayTyagi commented 1 month ago

ok thanks .

SaumayTyagi commented 1 month ago

Actually after compressing my file it still is 10MB and is causing problems . what to do?

ScriptTiger commented 1 month ago

Get a smaller hosts file. I'd recommend Steven Blacks' hosts files. You can also try disabling DNS caching.

SaumayTyagi commented 1 month ago

cant we use dnsmasq for more compresing or ipsets and iptables . actually i found this suggestion on some site . but i think dnsmasq and ipsets , iptables are for linux

ScriptTiger commented 1 month ago

Hosts-bl supports generating files in the following formats: adblock, dnsmasq, dualserver, fqdn, hosts, ipv6, privoxy, rfqdn, rpz, unbound. Just research which ones you want to use. For Windows, you can use Dualserver to set up a DNS server on your internal network rather easily. Privoxy is a proxy server and also easy to set up. You could set up all of them, or just one, whatever your needs are and whatever your comfort level is. Setting up multiple is usually recommended for a multi-layered approach.