Option to use PiHole for ad-blocking

[Ge0rges]

Is your feature request related to a problem? Please describe. This feature request is not related to a problem. It is an improvement.

Describe the solution you'd like Currently lightweight ad-blocking is a very useful option provided by Algo and done at the DNS level. PiHole is an open source ad-blocking solution that while not as light weight provides a better ad blocking solution, also at the DNS level, while providing extra features like tracker blocking.

Describe alternatives you've considered The alternative is the currently implemented lightweight solution. However using PiHole would allow more advanced users to tailor this ad blocking feature to their liking by customising PiHole.

[Rohmilchkaese]

You could just simply add pi holes ip in config.cfg. You need to allow the network range that algo hands out to vpn devices using ip tables. Works just fine. I added stubby for dns over tls on top.

[Ge0rges]

Isn't there quite a bit of setup required between Pihole and the wg0 interface, as well as firewall?

[Rohmilchkaese]

No, pihole runs on ethernet in my case. I only allowed incoming traffic from Wireguard vpn ip range. Make sure port 53 is closed if you planning to run this on an vps.

Edit: Listen on all interfaces in pi hole settings.

[TC1977]

Don't forget to open up port 80 on the Algo server (and the cloud firewall, if applicable) if you want to access the Pi-hole web interface.

[dan-v]

I added pihole to replace the existing dns adblocking on my fork (https://github.com/dan-v/algo) for anyone interested. Once deployed and connected to VPN you can just hit the URL http://pi.hole to get to admin UI. The ansible changes are a bit hacked in there at the moment, but if there was interest maybe could make this a bit more official and create a pull request. Right now I put no password on the pihole admin UI, but if this was a shared VPN like algo supports that would be a bit strange as everyone could see each others DNS requests, so not sure how best to handle that.

[magarto]

I really liked your fork dan-v, works flawless, thank you veryyy much. Would you keep it updated or commit changes with diff just in case trailofbits would consider to implement it?

[dan-v]

@magarto - it shouldn't be too hard to rebase future changes from upstream as I haven't made many modifications to the existing code base. I don't monitor the changes to algo very frequently so feel free to ping me on the forked repo if there are changes you see that look important that should be pulled in. I'd obviously prefer to just get support added here for Pi-hole though and not maintain a fork.

[dan-v]

Some random open questions worth considering if support for Pi-hole were to be considered:

• Should there be an option to choose between existing adblock solution or Pi-hole or should Pi-hole replace the existing solution? I prefer Pi-hole over the existing solution as it allows you to easily see all DNS requests and easily make modifications to the allow/block lists, which is really powerful especially on a mobile OS. That said, Pi-hole is a much heavier weight solution and a bit awkward in a multi user environment as there is just a single admin user that can view and modify DNS traffic.
• How should the Pi-hole admin UI be handled? Should a password be set? If so, should it be auto generated?
• The provided Pi-hole installer is currently a giant shell script and not sure how well that works with ansible's idempotent setup.

[magarto]

Some random open questions worth considering if support for Pi-hole were to be considered:

• Should there be an option to choose between existing adblock solution or Pi-hole or should Pi-hole replace the existing solution? I prefer Pi-hole over the existing solution as it allows you to easily see all DNS requests and easily make modifications to the allow/block lists, which is really powerful especially on a mobile OS. That said, Pi-hole is a much heavier weight solution and a bit awkward in a multi user environment as there is just a single admin user that can view and modify DNS traffic.

I think Pi-hole is ok. It is heavier than other solutions but very confortable. If any user needs a different solution he can just install dnsmasq/outbound... but it's up to you.

• How should the Pi-hole admin UI be handled? Should a password be set? If so, should it be auto generated?

I think so, "pihole -a -p" and the user makes and input or autogenerated with a random value and output the password together with the CA and shared keys. User could be questioned about adding a CGP firewall rule to make 80 and 443 accesible (in my case I do not have that, only when connected to the vpn).

[shapiro125]

This is really neat! thanks @dan-v -- i could never get pihole to work properly when installing separately. i'm using your fork, but it keeps stalling on me when it gets to configuring wireguard. has anybody seen anything like that? specifically, the step "WireGuard repository configured"

[dan-v]

@shapiro125 - i haven't bumped into that issue on the limited number of deployments I've done to DigitalOcean. What provider are you deploying on?

[shapiro125]

I'm using Vultr. I realized it was hanging trying to install Wireguard in the role right after pihole. To get it to work I added a pause to the Ansible code and installed Wireguard manually on the server. Doubt it's anything with your code -- I bet it's more likely something weird happening with memory or CPU on my VPS.

[Ge0rges]

@dan-v awesome fork. Would be cool to have some options for the Pihole configuration (privacy level, yes/no install of web interface...).

[axeII]

@dan-v make this please official.

[dmwyatt]

@L3rchal He can't make it official. Note that next to his name it doesn't say "member" or "contributor". He doesn't have anything to do with this repo.

Anyway, Pi-hole integration would be awesome!

[Ge0rges]

He could submit a pull request.

Sent from my iPhone

On Jan 22, 2019, at 12:21 PM, Dustin Wyatt notifications@github.com wrote:

@L3rchal He can't make it official. Note that next to his name it doesn't say "member" or "contributor". He doesn't have anything to do with this repo.

Anyway, Pi-hole integration would be awesome!

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub, or mute the thread.

[dan-v]

I'd like to hear some feedback from maintainers on whether they 1) even want to support this 2) if wanted, should it replace existing adblock solution or be an optional choice. Without that feedback it doesn't make sense to spend the time on a proper pull request.

[sinkingfeeling]

@dan-v In my opinion the best path would be to make it a choice. Why force something like Pi-Hole on someone if they have zero interest in ever logging in to the web UI. If someone wants to enable adblocking and not have to touch anything then the current Privoxy seems like a good option.

Thanks for all the hard work with your fork and I look forward to trying it! 👍

[shapiro125]

@dan-v -- i love your pihole fork, but i noticed that it crashes if dns encryption is set to false. it can never connect to the dns resolution service, retries, and then ultimately fails. The new refactoring PR https://github.com/trailofbits/algo/pull/1334 makes dns encryption false by default, so not sure if there's a setting that should be changed in the unattended install options. Wasn't able to submit this issue on your fork, so apologize for spamming this comment.

[dan-v]

@shapiro125 - yes, currently my fork is hardcoded to only work with dns encryption enabled. This is one of the things that would need to be cleaned up.

[Benjamin-Lee]

@dan-v, it appears that the refactoring done in #1334 has made your fork incompatible. Can you update it?

Thanks so much for the great work on integrating Pi-Hole!

[an3ki]

This is great @dan-v thank you so much for adding pihole, i hope you do end up updating or making a request.

[rodeodomino]

Figured, I'd share. I've been updating @dan-v's amazing work with new Algo releases for my own personal use (and adding a bunch of other things, like @mmoti's regex filters and updaters). I've also been experimenting with split tunnel over wireguard, but open to questions and issues here: https://github.com/rodeodomino/algo-pihole

Hopefully, can keep this updated with new Algo releases as they happen!

[taprobana]

@dan-v can't thank you enough for your fork Dan..

[simX]

bump

I used @rodeodomino’s branch above. But it would be nice to get this merged in to the main repo.

Any guidance about how this should be integrated with the existing ad-blocking solution? Should it just be an option?

[nitrohorse]

Definitely would want this option vs having a separate fork.

[jackivanov]

Could you make a scope of the things you miss in the current DNS ad-blocking solution and what PiHole can bring to the project, please?

[dan-v]

@jackivanov, for me personally PiHole is preferential as it is a much more flexible and easy to use DNS adblocking solution when used from a phone. I can easily view all DNS traffic on a simple webpage (or app) and then easily block domains as needed. For example, if I install a new unknown app, I can look at all outgoing DNS requests and block any unwanted DNS traffic on the fly. While there are non root firewalls on Android that can actually fully block traffic (not just DNS), they prevent you from also running with VPN as they internally use VPN to provide this network blocking capability. So VPN with PiHole is best I've found to cover this use case.

[TC1977]

As a total novice who runs both Algo and Pi-hole, on separate servers, I don't think integrating the two is a great idea. Pi-hole is a great project but very complex and top-heavy, and I've found that enabling DNSSEC can cause problems. All the convenient web management you mention involves opening up port 80 on the Algo server, which would be open to essentially anyone with a borrowed device or stolen client config file, and a web client with root capabilities. Also, it uses a heavily modified dnsmasq but doesn't offer DoH by itself (yet), so we'd also have to install dnscrypt-proxy or cloudflared. Meanwhile dnscrypt-proxy is very lightweight, handles both DNS encryption and adblocking, and is updated very frequently.

I'm running Pi-hole on my home network for adblocking, but pointing it to dnscrypt-proxy for DoH. When it started randomly crashing, I had to file issues with the pi-hole FTL team, pi-hole, the dnscrypt-proxy team, and ultimately (through a proxy) the dnsmasq mailing list to track it down and resolve it. I was amazed by the support, but that's a lot of complexity and I think few would have the time/inclination to chase such an issue.

It's up to the Algo team to decide what they want to do, but I like the original idea of a low-cost, easy-to-setup, secure, disposable VPN server. This project already is surprisingly complicated, with Ansible scripts to be run on macOS/multiple Linux flavors/WSL/Docker, two different VPN protocols being installed, customized support for 10 different cloud providers, support for local install using Ubuntu or FreeBSD, cloud-init/shell script, update-users script, adblocking/Apple excluded network options, etc. etc. I see Pi-hole as being a great idea for a home network, or even a cloud install with careful firewalling, but I think it's best done separately and not as part of the mainline.

It's not a bad idea to write a tutorial on how to install Pi-Hole with Algo.

[dan-v]

@TC1977 - I think these are all great points. I've lightly touched on some of these issues in previous posts, but this is a good deep dive. Pi-hole is definitely heavy weight in comparison to the existing solution and certainly adds complexity in terms of both implementation and maintenance for the project. I think in a single user VPN setup it fits nicely, but with a multi user VPN it is a bit awkward, even if properly locked down. One option I proposed already is to have Pi-hole be an option rather than a replacement, but that also adds complexity in dealing with things like DNS encryption that will be solved in different ways. Looking beyond the bloat and complexities, I do think VPN + Pi-hole capabilities when used from a mobile device is quite powerful. I have been running my fork of algo + Pi-Hole for almost a year now with no issues and based on traffic to that repo it appears there are a number of other interested users. That said, just because a number of people find algo + Pi-hole useful doesn't mean it is a good fit for this project and it may not align with the project's vision. It's also always important to consider how much complexity, attack surface, etc would be added with any change. In other words, if I were maintaining the project I may stray away from pulling in a change like this (hence my hesitation to offer up a proper PR almost a year ago). But if I'm just a user responding to an ask for things I miss in the current DNS adblocking solution, I'm voting for Pi-hole.

[moshesiegel]

@jackivanov, for me personally PiHole is preferential as it is a much more flexible and easy to use DNS adblocking solution when used from a phone. I can easily view all DNS traffic on a simple webpage (or app) and then easily block domains as needed. For example, if I install a new unknown app, I can look at all outgoing DNS requests and block any unwanted DNS traffic on the fly. While there are non root firewalls on Android that can actually fully block traffic (not just DNS), they prevent you from also running with VPN as they internally use VPN to provide this network blocking capability. So VPN with PiHole is best I've found to cover this use case.

~TLDR pi-hole lets me know if a site isn’t working because of ad-blocking which I can whitelist VS if my cloud IP is on some blacklist.

Just wanted to second this, the current Adblock solution is either on or off, which is more of a challenge when using while traveling and troubleshooting why a site isn’t working. Its what prompted me to use both @dan-v and @rodeodomino Pi-hole integrated install. Yes it does add a few more minutes To the install but the base pi-hole setup works out of the box and can be configured for DoH by default and ultimately I’ve found it no less “disposable” or set and forget.

[kfken]

I'm not sure "time to install" is the metric that is most relevant, but rather security and complexity. @TC1977 makes good points about the cost/benefit for the average user. It's more difficult to maintain support, and doesn't offer any additional security benefits (which is algovpn's chief goal, I would argue). It does add convenience for some use cases, but I wonder how often the average user actually needs those. If security is the goal of the project, achieving that with the least complexity should be prioritized.

I use pi-hole at home with dnscrypt-proxy, and it's great, and algo for travel, and have never had an issue where I needed to modify the blacklists in dnscrypt-proxy in algo. I know that mileage my vary, but since we're chiming in on UX, I wanted to throw in my two cents.

[rnsc]

@dan-v Thank you for your fork, even if it's a hack, it's definitely useful! I was always installing pi-hole separately, but integrating it into the algo playbooks is easier.

It'd be cool if the Algo could give an official statement regarding this request :)

[mgrimace]

@dan-v any chance you're taking a look at the Pi-Hole v5.0 integration? It broke the installation on @rodeodomino 's fork and I've been trying to take a closer look since. I edited their roles/pihole to remove the wildcard and listsscripts, and I was able to get it to install, but then it times out on "[✗] DNS resolution is currently unavailable" and the installation still fails. I'm not sure where to go from here myself. I think getting rid of some of the extra scripts/gravity tweaks helped (since they changed gravity in v5.0). Thanks both of you for your work!

[simX]

@mgrimace While there might be issues creating a direct installation with Pi-hole 5.0, I would use the existing fork, and then just ssh into the cloud provider you used (I use DigitalOcean), and just run the pihole -up command. That worked successfully for me, and Pi-hole 5.0 is running great with my existing installation of algo.

[mgrimace]

@simx thanks, that actually worked for me to update my existing algo+pi-hole. However, I noticed some issues with whitelisting (it wasn't working and I was getting queries blocked by regex even with all my regex rules deleted)... so in short, I deleted the server instance (EC2 on AWS) to start fresh and haven't been able to re-install (and @rodeodomino archived their fork).

[moshesiegel]

I’ve got the same issue with DigitalOcean and Vultr.

Thanks, Moshe —

On May 25, 2020, at 7:56 PM, mgrimace notifications@github.com wrote:

@simX thanks, that actually worked for me to update my existing algo+pi-hole. However, I noticed some issues with whitelisting (it wasn't working and I was getting queries blocked by regex even if deleted)... so in short, I deleted the server instance (EC2 on AWS) to start fresh and haven't been able to re-install (and @rodeodomino archived their fork). — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[dan-v]

I haven't looked at this in a while but have been using it everyday. I'll take a look at pulling in latest algo and getting pihole working when I have some free time.

[moshesiegel]

Thank you! Working on DigitalOcean. Failed with Vultr

Thanks, Moshe —

On May 26, 2020, at 4:46 PM, Dan Vittegleo notifications@github.com wrote:

 @mgrimace - I pulled in latest algo master and added latest Pi-hole: https://github.com/dan-v/algo. It required a few tweaks to get everything working again. The Ansible changes are still hacked in there like before; just enough to get it running basically. Note this setup requires DNS encryption to be enabled and you need to choose to enable adblock during setup. I've only tested it out on DigitalOcean.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or unsubscribe.

[dan-v]

@moshesiegel - made one more commit to fix an issue with dhcpd preventing all network connectivity on reboot, so might want to redeploy.

I pulled in latest algo master and added latest Pi-hole: https://github.com/dan-v/algo. It required a few tweaks to get everything working again. The Ansible changes are still hacked in there like before; just enough to get it running basically. Note this setup requires DNS encryption to be enabled and you need to choose to enable adblock during setup. I've only tested it out on DigitalOcean.

[mgrimace]

Thanks so much @dan-v! I just tried deploying it to AWS. Unfortunately same error at the pihole install task, it times out with "[✗] DNS resolution is not available"

[dan-v]

@mgrimace - i likely won't try to get this working on anything other than DigitalOcean as that's where I'm running it. The whole DNS situation here is kind of a giant mess unfortunately as there's a bunch of competing software and that makes this fragile. Pi-hole wants to run it's own DNS server using dnsmasq on port 53 and disables the default resolver systemd-resolved that runs on port 53 and the dnscrypt-proxy OS package by default uses systemd sockets and wants to binds to port 53.

[moshesiegel]

New commit is working just fine on DO including DHCP after reboot.

Thanks, Moshe — Cell 312-331-0229

On May 27, 2020, at 1:02 AM, Dan Vittegleo notifications@github.com wrote:

 @mgrimace - i likely won't try to get this working on anything other than DigitalOcean as that's where I'm running it. The whole DNS situation here is kind of a giant mess unfortunately as there's a bunch of competing software and that makes this fragile. Pi-hole wants to run it's own DNS server using dnsmasq on port 53 and disables the default resolver systemd-resolved that runs on port 53 and the dnscrypt-proxy OS package by default uses systemd sockets and wants to binds to port 53.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe.

[mgrimace]

@dan-v hey thanks for doing what you've done already to get it working on DigitalOcean! This is above my technical skill level, if anyone has any ideas to get it working on AWS I'd appreciate any help! Thanks again

[rodeodomino]

One solution would be swapping out the ppa dnscrypt-proxy package for the go version, but it breaks apparmor: https://github.com/trailofbits/algo/pull/1747. That version disables the systemd-socket.

[dan-v]

One solution would be swapping out the ppa dnscrypt-proxy package for the go version, but it breaks apparmor: #1747. That version disables the systemd-socket.

Nice, yeah that would certainly be an improvement.

[mgrimace]

Thank you @rodeodomino, and thank you also for your work with the previous fork I made great use of it! If anyone is able to make these changes, I'd be happy to test it out with AWS and report back!

[gfitzp]

@mgrimace: I tried @dan-v's fork but it didn't seem to want to install (not sure if it was because of a Digital Ocean-specific setup or if other changes had broken it), and it looks like the PPA dnscrypt-proxy is no longer being used on Ubuntu 20.04 since this past August (8b2b57dedaaa8fdb69caed1d29237815311ab41e), so I made a fork of the current Algo that's working for me when installing to an existing Ubuntu 20.04.1 server. It includes a toggle in the config.cfg file to enable Pi-hole support as long as you also enable DNS adblocking when prompted, if you want to see if it works with AWS: https://github.com/gfitzp/algo

Edit: I may have spoken too soon; it works, but unable to resolve DNS after a reboot. I'll keep working on it. Ok, so I think on startup Pi-hole was trying to forward DNS queries to dnscrypt-proxy, but dnscrypt needed a fallback DNS server IP to use to do an initial download of its list of servers, and without that it couldn't start. Also incorporated @dan-v's code to remove the DHCP server, and @rodeodomino's code to explicitly start and restart the Pi-hole service on server boots and restarts, as well as to display the client name in place of the client's IP address in Pi-hole. Hopefully I've got it working properly now, but I admit a lot of what I've done has been trial and error. :)

[mgrimace]

That's incredible @gfitzp! I was never able to problem solve the DNS issues and it sounds like you've got it working! I ended up doing it all from scratch/manually using PiHole, PiVPN (with WireGuard split-tunnel), and Unbound on AWS. I put together my install notes/guide here: https://github.com/mgrimace/PiHole-with-PiVPN-and-Unbound-on-VPS-. It's not in script format (I don't know how to do that) but I tried to keep the instructions as simplified as possible for others. So far no problems, but if something happens with PiVPN and I swap back to Algo I'll give yours a shot. Thanks for thinking of me here and appreciate the update!!

[adrianocr]

So this is probably a stupid question but why can’t the algo script just be set up to take a dns address? That way someone can install Pi-hole on a home server or a vps (or in a docker container in the same vps algo used) and you just provide the script with the ip. That way if someone wanted to use 8.8.8.8 or 1.1.1.1 or whatever else (like Pi-hole) they could just input whatever address they need.