opnsense / plugins

OPNsense plugin collection
https://opnsense.org/
BSD 2-Clause "Simplified" License
834 stars 621 forks source link

os-bind - Thoughts Features #974

Closed rhysxevans closed 5 years ago

rhysxevans commented 5 years ago

Hi

Firstly, I would like to say thank you, for your work on this, it is appreciated. However I would like to see the following added to the os-bind plugin

1) Host overides - basically move away from the Unbound package and solution "chaining" 2) Ability to enable safe search features (google, bing, youtube etc) from a check box type setup (I am currently doing this via unbound and the custom options) ( https://www.cwssoft.com/?p=1577 , https://forum.netgate.com/topic/105598/using-bind-to-enforce-google-safesearch/4) 3) Ability to identify blocking lists - Ie www.xxx.com was blocked by top porn list (bad example I know) 4) MalwarePatrol list integration - this requires people to sign up, and each user is assigned a unique reciept key which will need to be factored in (https://www.malwarepatrol.net/tech-support/ & https://www.malwarepatrol.net/non-commercial/). I am particulary interested in the ransomware list, followed by the URL block Lists (they are quite agreesive and would require whitelist management) 5) Ability to do RPZ lists via a master slave bind DNS setup (OPNsense being the slave) (https://www.deteque.com/app/uploads/2018/04/Spamhaus-DNS-RPZ-DROP-Setup-v.2.pdf & https://deepthought.isc.org/article/AA-00518/0/How-can-I-synchronize-DNS-RPZ-firewall-policies-across-multiple-DNS-servers.html & https://www.securityzones.net/images/downloads/BIND_RPZ_Installation_Guide.pdf)

I have previously done most of the above on linux variants before, however this has always been via CLI, and I wouldn't have the foggiest on how to present this via the GUI. So basically I am willing to help on the backend side, where/if required

Thanks

mimugmail commented 5 years ago

I'm planning full authoritative DNS for next year, but for interim, Overrides should be doable. Force safesearch should be easy, will add it.3. is not possible since all lists will be put in one file and kill duplicates. Malwarepatrol is a good idea, I have an account somewhere .. will have a look. :)

rhysxevans commented 5 years ago

Thanks, let me know, if you would like me to help out or test anything

rhysxevans commented 5 years ago

Sorry, I forgot 1 thing from my list, not a major thing.

The ability to specify a custom rpz response, allowing for a walled garden type scenario where users are redirected to a page that will tell them that the site has been blocked

mimugmail commented 5 years ago

With a separate RPZ zone, you mean just a new tab with key => value putting this to a zone like within telegraf:

image

rhysxevans commented 5 years ago

Hi

I may be being dumb (more than likely) , which feature above is this related to ?

I thought you implied everything went into a single RPZ zone for the blacklists , multiple zones would allow for list differentiation of source logs in the logs ?

If it is regarding the walled garden question, that could be dealt with just by setting the "default" policy response to a CNAME that can be entered via the gui

Thanks

mimugmail commented 5 years ago

No .. not a single file, only the blacklists. I'm hoping you can read named.conf, perhaps you'll then understand my question:

acl "lan" { 192.168.0.0/24; 127.0.0.0/8; };

options {

        directory       "/usr/local/etc/namedb/working";
        pid-file        "/var/run/named/pid";
        dump-file       "/var/dump/named_dump.db";
        statistics-file "/var/stats/named.stats";

         listen-on port 53530 { 127.0.0.1; };
         listen-on-v6 port 53530 { ::1; };

        response-policy { zone "whitelist.localdomain"; zone "blacklist.localdomain";zone "rpzgoogle";zone "rpzyoutube"; };

        recursion          yes;
        allow-recursion    { lan; };

        max-cache-size    80%;

        dnssec-validation    no;

};

key "rndc-key" {
        algorithm hmac-sha256;
        secret "VxtIzJevSQXqnr7h2qerrcwjnZlMWSGGFBndKeNIDfw=";
};
controls {
        inet 127.0.0.1 port 9530
                allow { 127.0.0.1; } keys { "rndc-key"; };
};

zone "." { type hint; file "/usr/local/etc/namedb/named.root"; };

zone "localhost"        { type master; file "/usr/local/etc/namedb/master/localhost-forward.db"; };
zone "127.in-addr.arpa" { type master; file "/usr/local/etc/namedb/master/localhost-reverse.db"; };
zone "0.ip6.arpa"       { type master; file "/usr/local/etc/namedb/master/localhost-reverse.db"; };

zone "whitelist.localdomain" { type master; file "/usr/local/etc/namedb/master/whitelist.db"; notify no; check-names ignore; };
zone "blacklist.localdomain" { type master; file "/usr/local/etc/namedb/master/blacklist.db"; notify no; check-names ignore; };
zone "rpzgoogle" { type master; file "/usr/local/etc/namedb/master/google.db"; notify no; check-names ignore; };
zone "rpzyoutube" { type master; file "/usr/local/etc/namedb/master/youtube.db"; notify no; check-names ignore; };

logging {
        channel default_log {
                file "/var/log/named/named.log" versions 3 size 7m;
                print-time yes;
                print-severity yes;
                print-category yes;
        };

        channel query_log {
                file "/var/log/named/query.log" versions 3 size 7m;
                print-time yes;
        };

        channel rpz_log {
                file "/var/log/named/rpz.log" versions 3 size 7m;
                print-time yes;
        };

        category default { default_log; };
        category general { default_log; };
        category queries { query_log; };
        category rpz { rpz_log; };
};
rhysxevans commented 5 years ago

Hi

Ok, so your talking about the safe search stuff the "zone "rpzgoogle"" and "zone "rpzyoutube"" in this case ?

If so the proposed above would work.

I assume Services -> Bind -> Configuration -> SafeSearch Tab -> tick boxes ?

Thanks

mimugmail commented 5 years ago

No, I mean the walled garden thing. Can you add a sample named.conf and a zone how you would use a walled garden. Perhaps I'm intepreting walled garden in a different way than you do :)

rhysxevans commented 5 years ago

Ah ok, so there are 2 ways to do this on each entry in the RPZ zone file or at the policy definition (Sorry my terminology may be wrong)

Within the zone file on a per line basis you could do - Don't think this is the way you want to go

$ORIGIN rpz.example.net.
evil.example    CNAME   walled.example.
evil2.example   CNAME   walled.example.

or at the policy definition with something like - I need to verify the exact syntax (will try do this today)

responsepolicy { 
        zone "white" policy PASSTHRU;
        zone "black" policy CNAME walled.garden;
}

CNAME could be replaced with A - but I think CNAME is more flexible, as it allows you do do some stuff on the web server side (out of scope) to identify source urls etc

This is probably a better description of a walled garden than I can do https://kb.isc.org/docs/aa-00520

Thanks

mimugmail commented 5 years ago

OK, so how you describe it lokal Overrides and walled Garden could be in the same file

rhysxevans commented 5 years ago

Hi

I would presume they would be different, however a walled garden may require a host override for the CNAME entry.

1) Basically in RPZ blacklist configuration, setup required redirect CNAME 2) Setup a Host Override for the CNAME created above (this could be on public name servers aswell)

Hopefully I am making sense

Thanks

rhysxevans commented 5 years ago

Hi

Just confirmed the syntax, slight typo in the above example (responsepolicy should be response-policy)

So

response-policy { 
        zone "white" policy PASSTHRU;
        zone "black" policy CNAME walled.garden;
}

But essentially the format is correct, and walled.garden could be any valid DNS entry

Thanks

mimugmail commented 5 years ago

Ok, I belive I know how to implement it. It should be finished next week. Have to check some constraint options with @fichtner first :)

rhysxevans commented 5 years ago

great, thanks, let me know if you need me to do anything

rhysxevans commented 5 years ago

Hi,

Just updated and have flipped over to BIND for SafeSearch , working like a charm.

Thank you

mimugmail commented 5 years ago

Next update brings also safe search for Bind, then I'll start with Overrides ...

stilez commented 5 years ago

I'd like to request a small feature in the bind configuration page.

As users and use-cases vary a lot, it's often helpful to have a text box for "custom config" the user can enter, which gets appended to any config genberated by the system. Unbound has this, and it's incredibly useful. I'd like to add some Bind config that isn't common use, so the UI standard fields won't handle it.

If Bind plugin had an "enter any custom config" text box, I could do it all from the GUI. Instead I need to do it via CLI, and save a copy because it'll surely be deleted when named.conf is regenerated/recreated, or the platform updates.

As no real processing is needed (accept text in a text box, append it unmodified to named.conf when this is generated), this seems very easy. Any chance? It would put Bind on parity with Unbound for less-common config entries - and there are quite a few things that Unbound can't do, and Bind can.

(Use case: I'm using Unbound as my main local resolver, but I need to allow a single domain (only) to be resolved with Bind on the WAN, for Let's Encrypt DNS-01 validation purposes only . To ensure this is locked down, I'm using Unbound as LAN-facing resolver (only), and Bind as a very limited WAN-facing resolver (only) able to resolve just a single ACME subdomain only and be locally updated via certbot/RFC2136. This is way too specialised to handle via usual UI fields, also as I can lock down far more than usual since the use is so narrow, so entering custom named.conf lines will be needed)

mimugmail commented 5 years ago

@stilez can you post your exact config please? Is this some RPZ related rewriting what you need?

stilez commented 5 years ago

@stilez can you post your exact config please? Is this some RPZ related rewriting what you need?

I'm still working it out, it's going to take a while to get good. Tentatively, it might look like this:

options {
        directory       "/usr/local/etc/namedb/working";
        pid-file        "/var/run/named/pid";
        dump-file       "/var/dump/named_dump.db";
        statistics-file "/var/stats/named.stats";

    // service hardening

    version none;
    hostname none;
    rate-limit {
                responses-per-second 3;
                // exempt-clients { 127.0.0.1; ::1; };   // NONE NEEDED AT PRESENT
        };

    // authoritative server only

        recursion no;
        fetch-glue no;
        allow-transfer { none; };
        allow-query { localhost;LETSENCRYPT'S_IP_OR_SUBNET_FOR_QUERYING; };

    // general service definitions

        listen-on port ******* { localhost;LETSENCRYPT'S_IP_OR_SUBNET_FOR_QUERYING; };
        dnssec-validation    auto;  // TO SET UP LATER
};

logging {
        [ default OPNSense config]
};

// "." ROOT ZONE HINT - should be OMITTED/COMMENTED OUT for authoritative server

zone "acme-validation.example.com" {
    type master;
    file "/etc/bind/zones/db.acme-validation.example.com";
    // allow-transfer { IP_ADDR_TO_ADD ; };  // ENABLE WHEN SLAVE NS EXISTS
        notify yes;
        [ OTHER ZONE DATA / FILE LINK]
};

My scenario: I want to use Let's Encrypt acme/certbot DNS-01 in "DNS Alias" mode (described here), as acme/certbot can't use my ISP's DNS API. The domains are private use only - I only need DNS for myself, when I'm accessing them remotely via HTTPS (and need the TLS cert chain to work from other locations), nobody else uses them. Certbot can be used via RFC2136 with Bind though, so if I create a "dummy/fake" domain (acme-validation.mydomain.com), redireect the usual validation domain from my ISP to Bind's public IP, and use Bind locally as the authoritative NS for the acme validation domain, I can get acme/certrbot working nicely.

The only time Bind should get queried is once every 60 - 90 days, when Let's Encrypt wants to check the TXT records for acme-validation.mydomain.com. There are no other records, and the domain doesn't actually exist anywhere outside Bind.

Because that's a very narrow use-case, I can lock down Bind completely. I am using Unbound for my local resolver and want to continue doing so, listening on the LAN interface only. I enable Bind on port 53 of an unused public IP on the WAN interface with no recursion/forwarding, no root zone hint, heavy rate limiting, and one zone defined only, a bit like above. And with luck - acme/Let's Encrypt will now work, even though my ISP's DNS API isn't compatible. But it's a very customised config, not a common use-case (or is it?) so I can't see myself doing this purely using the OPNSense GUI. I need to be able to manually create named.conf content lines.

mimugmail commented 5 years ago

Because your setup is so specific it should be easy to implement. What is the reason for excluding hint file? As you disallow recursion and only allow your provider to query?

And what are the contents of your zonefile?

stilez commented 5 years ago

What is the reason for excluding hint file? As you disallow recursion and only allow your provider to query? And what are the contents of your zonefile?

The hint is disabled based on this ISC technical note, which says to separate caching and authoritative nameservers, and explains the implication of a root hint for them.

As no other NS will be updated in a timely manner with the ACME TXT validation data (or updated at all), there's no point in providing a hint to an external NS, and it's detrimental in the context - at best it will cause Let's Encrypt's system to look at other NS outdated versions of the TXT data if for any reason it doesn't reach Bind, rather than note the failure or try again, causing an explicit validation error (wrong TXT data found) rather than a retry or "can't connect" error, and may cache the bad result as well.

I don't yet know the zonefile contents I will need - I'm figuring this out and need to experiment, as I'm new to Bind and it also depends on ACME+ RFC2136 plugins (using Bind because of ISP/provider Let's Encrypt limitations, otherwise I'd do it via ISP/provider API + ACME, in which case I wouldn't run Bind at all).

mimugmail commented 5 years ago

Ok, got it. Let's do it the other way around. What exactly to you need drom LE DNS-01 to work. Is there a technical doc from LE which explains for every DNS software how to configure? Maybe this way I can build a workaround for you.

My LE knowledge is sadly near 0 :)

stilez commented 5 years ago

To avoid jamming this thread with a slight side topic, let's discuss LE and its DNS-01 system by email, so we can focus here, on Bind plugin features/enhancements. We can post back here, any conclusions or thoughts. I've sent you a PM on the OPNsense forum with my email. When I get a reply, I'll email you what I know about it. It's not complicated :)

ghost commented 5 years ago

Thanks for the work on the BIND package - I have it working nicely also :-)

Just wanted to add support for a couple of the feature requests:

Thanks.

macktic commented 5 years ago

Thanks for all the hard work. I’ve read all above, but would still request a custom option somehow. Even if it would be in a file on the file system and only available via ssh. I’m currently adding options that way, but they get deleted on save or update.

Anyway, the option I’m after is to strip all IPv6 answers ( filter-aaaa-on-v4 yes; ) I don’t need bind to listen on an IPv6 address either, but that’s currently not possible in the gui either, as I’ll get an error if I don’t put anything in the listen IPv6 address.

Some background: My ISP doesn’t support IPv6 at the moment, so I’ve got IPv6 functionality via HE net tunnelbroker.

The issue is that Netflix detects this as a possible way to get around their blocks and stops working.

The easiest way round this is by stripping all IPv6 answers from any of the Netflix servers.

I do this (as I used to on pfsense) by using unbound as normal dns server and use host overrides to send the Netflix servers to bind (running on a non standard port) in this scenario I don’t need bind to give me any IPv6 answers ever.

mimugmail commented 5 years ago

So, all you need is to make v6 field optional and a checkbox to strip v6 answers? Thats easy :)

Next version will also offer zone management

L1ghtn1ng commented 5 years ago

@mimugmail Would be good if we could have requests go over DoH using cloudflare https://developers.cloudflare.com/1.1.1.1/dns-over-https/

mimugmail commented 5 years ago

Does bind support DoH yet? Perhaps we need a stunnel plugin ;) https://kb.isc.org/docs/aa-01386

L1ghtn1ng commented 5 years ago

Not that I am aware of but using a forwarder that would somehow point to cloudflare would work right? /home/pi/argo-tunnel/cloudflared proxy-dns --port 54 --upstream https://1.1.1.1/.well-known/dns-query -- upstream https://1.0.0.1/.well-known/dns-query --upstream https://2606:4700:4700::1111/.well-known/dns-query --ups

Having to use this on my pihole currently to have the DoH support and just modified the config to point to that

Just would be nice to have a solution with no workarounds

On Fri, 2019-05-03 at 17:56 +0000, Michael wrote:

Does bind support DoH yet? Perhaps we need a stunnel plugin ;) https://kb.isc.org/docs/aa-01386 — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

mimugmail commented 5 years ago

Why not using dnscrypt proxy plugin? I'm using it as main DNS on Firewall, also with DNSBL support.

L1ghtn1ng commented 5 years ago

Can you create a how to for it please?


From: Michael notifications@github.com Sent: Saturday, May 4, 2019 7:21:12 AM To: opnsense/plugins Cc: J.Townsend; Comment Subject: Re: [opnsense/plugins] os-bind - Thoughts Features (#974)

Why not using dnscrypt proxy plugin? I'm using it as main DNS on Firewall, also with DNSBL support.

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/opnsense/plugins/issues/974#issuecomment-489298805, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AA3V2QXZQE5FY663CRVIXPTPTUTNRANCNFSM4GDFOYRQ.

mimugmail commented 5 years ago

Next time you try it on your own ;) https://github.com/opnsense/docs/pull/169

L1ghtn1ng commented 5 years ago

Thank you 🤗


From: Michael notifications@github.com Sent: Sunday, May 5, 2019 6:35:27 PM To: opnsense/plugins Cc: J.Townsend; Comment Subject: Re: [opnsense/plugins] os-bind - Thoughts Features (#974)

Next time you try it on your own ;) opnsense/docs#169https://github.com/opnsense/docs/pull/169

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/opnsense/plugins/issues/974#issuecomment-489446988, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AA3V2QWTVMNKYEMDAHDOUC3PT4LF7ANCNFSM4GDFOYRQ.

mimugmail commented 5 years ago

1, 2, 3, 5 done, close?

fichtner commented 5 years ago

Sure, 4 can be re-requested.

zanechua commented 4 years ago

@mimugmail was the option for ipv6 filtering ever added in?

For filter-aaaa-on-v4? I don't see it in the UI for opnsense for the most part.

Does this depend on https://github.com/opnsense/ports ? I see the filter-aaaa option inside the bind911 Makefile but I don't see it in the bind 914 Makefile

mimugmail commented 4 years ago

Not that I'm aware of, how is the exact Syntax?

zanechua commented 4 years ago

It just needs to be enabled during the binary build with ./configure --enable-filter-aaaa

And usage is just simply

filter-aaaa-on-v4 yes;
filter-aaaa-on-v6 yes;

Docs: https://kb.isc.org/docs/aa-00576

zanechua commented 4 years ago

@mimugmail

Actually. Looking at the Makefile, the os-bind plugin is using bind914.

bind916 is the next ESV version after bind911. Curious why bind 911 isn't being used since it's EOL in 2021 Q4 while bind914 is EOL 2020 Q1 and bind916 is EOL 2024 Q1.

Any issues reverting to bind911 since that will still be supported? bind911 has the filter-aaaa as default build options.

I've already submitted a pull to add it into bind916 upstream.

Bind support timeline: https://kb.isc.org/docs/aa-00896

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=244332

EDIT: Marek over at the freebsd bugzilla pointed out to me that you are able to use the filter-aaaa as a plugin now without the need for building it. It just needs to be configured in named

Created a pull request to facilitate filter-aaaa:

https://github.com/opnsense/plugins/pull/1722