Adblocking on DJA0230 - Githubissues

jmytch commented 3 years ago

Are packages got something to do with opkg CLI or something? If that is so, can this Adblock be installed with it or not? https://github.com/openwrt/packages/blob/master/net/adblock/files/README.md

I'm new to these packages thing/opkg whatever they are called, the newbie have no idea where to start, any website you got would come in handy. Is it like addon, or something at the router level? I wonder if they show GUI cards after installing packages?

seud0nym commented 3 years ago

Are packages got something to do with opkg CLI or something?

Yes.

If that is so, can this Adblock be installed with it or not?

Probably. But it depends on how much it relies on standard OpenWRT configuration... Unfortunately, the only way to find out is to install it and see how you go.

Is it like addon, or something at the router level?

It looks like it re-configures dnsmasq, so at the router level, but it is an add-on.

I wonder if they show GUI cards after installing packages?

No. It is a standard OpenWRT package, so if we were on a standard OpenWRT release with the LuCI GUI, then yes you would be able to get configuration through the UI. But sadly not in these Technicolor versions of OpenWRT.

jmytch commented 3 years ago

Alright I'm going to give Adblock Package ago on the router level which would block ads across all devices.

Where do I manually upload those ipk file to? To /var/tmp/ or is it other directory?

opkg install adblock or would it be opkg install /tmp/adblock?

seud0nym commented 3 years ago

You can install it through the GUI now. Just go into the Packages card, set Custom Feeds on, and save. If you don't enable custom feeds, it won't be able to install the package.

When you have saved it, you will have the option to "Update Now". Tick the check box and save again. Once it has completed, the Available tab will have the available packages. Tick adblock and save, and it will install it.

jmytch commented 3 years ago

Adblock requires LuCi-app-adblock as they are not in "Available" tab for some reason. Are you able to add that?

seud0nym commented 3 years ago

No, I can't. Packages are built by a maintainer for a particular platform. We are lucky that someone has built packages that work on our devices. If a package isn't in the repository they have built, then it would be hit and miss trying to find one that will work.

And a LuCI package will never work, because it will have a dependency on the LuCI GUI, which is the OpenWRT standard but not installed on our devices.

If you are after adblocking, you could try Adguard Home since you have a DJA0230. It should have enough memory. See this post: https://forum.openwrt.org/t/howto-running-adguard-home-on-openwrt/51678

jmytch commented 3 years ago

That's look interesting, so if my router says it running armv71 I get the ARMv7 of 32-bit? Capture

seud0nym commented 3 years ago

Yes, 32 bit ARMv7 should be the correct one. But I wouldn't install the Beta or Edge versions - try just running the installer.

jmytch commented 3 years ago

Sorry what do you mean installer? The website you sent to me in other reply say to download the release version. Do you mean to install it by /opt/AdGuardHome/AdGuardHome -s install? I tried that but it never show up the AdGuardHome webpage, so I'm trying a different way like go to /opt/AdGuardHome/AdGuardHome then once complete, visit the url by going to 192.168.0.1:3000/install.htm for example as it should show up the webpage for installation

jmytch commented 3 years ago

Also I'm confused with this part. Linux_armv7 is what i should select? According to this post on whirlpool by a user, he confirms DJA0230 is ARMv7 - whrl.pl/Rf8bW3

Capture

seud0nym commented 3 years ago

Yes, it definitely should be ARMv7.

However, I just tried it on my test DJA0231 which is running 20.3.c, and it core-dumped, causing the router to reboot. Looks like there is some incompatibility.

I have tried Adguard Home before but that was on 18.1.c.

jmytch commented 3 years ago

Yeah when I run ARMv7 it said core-dumped in the PuTTY and cause router to reboot. Looks like it doesn't work for DJA0230 neither

seud0nym commented 3 years ago

Sorry I wasn't able to help.

FYI, I run Pi-hole ad-blocking on a Raspberry PI Zero-W. It is powered by the USB port of my DJA0231.

jmytch commented 3 years ago

Nah it's all good :)

I've wrote this to AdGuard Home, and I got a reply saying to not use nand for config file store (i used /tmp as workaround, but an usb device probably is better) and disable dnsmasqd which is not ideal as we are all using this? I know it taking the port 53, but that person who replied doesn't get what I was saying, there basically no AdGuard install web page to show up. So to put AdGuardHome in /tmp folder but doesn't /tmp delete the files when rebooting the router? Kinda weird doing that way.

https://github.com/AdguardTeam/AdGuardHome/issues/3283

seud0nym commented 3 years ago

Yes, /tmp will get cleared on reboot. I think FrancYescO was just suggesting it as a proof of concept (he is one of the developers who works on tch-nginx-gui).

Also the DJA0230 only has a very small storage area (89Mb and the AdGuardHome executable is about 19Mb), so a USB stick is the way to go.

You will need to run the ./update-ca-certificates script, otherwise AdGuard won't be able to download the filter lists. Check that it installed/updated the certificates correctly by running logread -e rpc.gui.opkg.

You will need to run the following commands to free up port 53:

uci set dhcp.main.port='53535'
uci set dhcp.guest.port='53535'
uci commit dhcp
/etc/init.d/dnsmasq reload

Then you should be able to install AdGuard as a service and get to the installation screen on http://192.168.0.1:3000.

You will need to set the web interface to a different port than 80 (I suggest 8008).

I have had a quick go at most of these steps (using the ARMv5 version as you suggested) and it seems to be working.

jmytch commented 3 years ago

Doesn't DJA0230 have DNS forwarding, where you forward DNS 192.168.0.1 to AdGuard Home address? However, I did tried setting it as DNS server but it didn't work. I must be missing something lol, need a clear instruction for DJA0230 and DJA0231. I'm pretty excited to use it

seud0nym commented 3 years ago

You have replaced the DJA0230 DNS server/forwarder with AdGuard Home by changing those dnsmasq ports to 53535 and runnning AdGuard Home as a service. AdGuard now listens on the DNS port (53) instead of dnsmasq.

On the Local Network screen, you need to set the DJA0230 as the DNS server, and it should all just work.

You can test it from your Windows PC by opening a command window and issuing this command: nslookup www.google.com 192.168.0.1

The 192.168.0.1 on the end forces it to use your DJA0230 (in case the DHCP DNS change hasn't yet propagated). Leave it off if you run ipconfig /all on your PC and it shows 192.168.0.1 (or the fe80: IPv6 address of the DJA0230).

You should get the response and then be able to check it in the AdGuard Home GUI (http://192.168.0.1:8008 if you followed my suggestion) by clicking on the number next to DNS Queries under General Statistics.

The DJA0231 may not have enough RAM.

seud0nym commented 3 years ago

Have you had any success with your AdGuard Home installation? I have been running it on my test DJA0230 for about 1 day. I loaded up the same filters I use on my Pi-hole servers (shows blocking around 390,000 sites, although the Pi-hole reports 730,000?).

I have hit it with a DNS tester and even though it is not an equal test (my Pi-hole uses a local unbound server as upstream DNS, whereas AdGuard Home is using Quad9), AdGuard Home seems marginally faster. But the free memory on the DJA0230 is now down to 23MB...

jmytch commented 3 years ago

Uh, not really. Might need to make a video tutorial on how to do that by recording software to show how its done.

DNS, Local network is already by default are using DJA0230 as shown.

DNS Server in internet access card should be 192.168.0.1 right?

Lastly, like you suggest a usb stick in the modem, I'm not entirely sure as I had never used usb stick on a router before. Plugging it in, but then when it comes to package, install onto usb if I'm understanding correctly? And due to file size where ad block builds up over time? I have a 16gb, 32gb and 64gb stick here. 16gb is more than enough surely?

With AdGuard Home using Quad9, I'm aware of that, I was going to use Cloudflare DNS over Quad9 might be the better option.

seud0nym commented 3 years ago

Sorry it has taken so long to get back to you - I have been playing with AdGuard Home and working out what we need to do to use it on our devices.

So, I have put up some scripts to do the configuration for you. You can find them here if you are interested: https://github.com/seud0nym/tch-gui-unhide/tree/master/adguard

I have run it a couple of times on different devices, and it seems to work, but I haven't tried it for any length of time.

The only gotcha is that if you are using the Guest Wi-Fi/network, then it isn't viable. I couldn't get dnsmasq working just as a DHCP server, and the AdGuard Home DHCP server doesn't deal with multiple address pools. But if you aren't using Guest Wi-Fi, it seems to be fine.

jmytch commented 3 years ago

Alright thank you for the script/guide, I'll try this over the weekend at some point. I reckon 16gb flash drive is more than enough for the adblock hey?

seud0nym commented 3 years ago

I have been testing on a 1GB flash drive, so I would say yes :-)

jmytch commented 3 years ago

Tried the Adguard today, unfoetunately it broke my connectivity on FTTN, tried uninstalling and didn't re enable dnsamq. Reboot router and now it seems to be bricked. Unable to access router at all

jmytch commented 3 years ago

From my PuTTY root@mymodem:~# curl -skL https://raw.githubusercontent.com/seud0nym/tch-gui-unh ide/master/adguard/agh-setup | sh -s -- [AdGuard-Home-Setup] Downloading latest version of update-ca-certificates... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 2324 100 2324 0 0 7085 0 --:--:-- --:--:-- --:--:-- 7063 [AdGuard-Home-Setup] Successfully downloaded update-ca-certificates. [AdGuard-Home-Setup] Running update-ca-certificates... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 113k 100 113k 0 0 60194 0 0:00:01 0:00:01 --:--:-- 60194 rpc.gui.opkg: Upgrading ca-bundle on root from 20190110-2 to 20210119-1... rpc.gui.opkg: Configuring ca-bundle. % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 123k 100 123k 0 0 65229 0 0:00:01 0:00:01 --:--:-- 65196 rpc.gui.opkg: Package ca-certificates (20210119-1) installed in root is up to date. [AdGuard-Home-Setup] Downlading latest version of intercept-dns... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 11193 100 11193 0 0 33918 0 --:--:-- --:--:-- --:--:-- 33918 [AdGuard-Home-Setup] Successfully downloaded intercept-dns. [AdGuard-Home-Setup] Downloading latest release of AdGuard Home... % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 9969k 100 9969k 0 0 3089k 0 0:00:03 0:00:03 --:--:-- 3089k [AdGuard-Home-Setup] Disabling dnsmasq to free up RAM and port 53... [AdGuard-Home-Setup] Configuring IPv4 WAN DNS... [AdGuard-Home-Setup] Configuring IPv6 DNS... [AdGuard-Home-Setup] Applying network changes... [AdGuard-Home-Setup] Configuring dnsmasq... [AdGuard-Home-Setup] dnsmasq disabled and stopped [AdGuard-Home-Setup] Freeing port 443... [AdGuard-Home-Setup] Reported Free RAM = 268804KB [AdGuard-Home-Setup] Extracting IP addresses from current configuration... [AdGuard-Home-Setup] Configuring AdGuard Home... [AdGuard-Home-Setup] Checking AdGuard Home configuration... 2021/07/10 10:38:45 [error] Couldn't get logging settings from the configuration: yaml: line 117: mapping values are not allowed in this context 2021/07/10 10:38:45.728895 [info] AdGuard Home, version v0.106.3 2021/07/10 10:38:45.731730 [info] Couldn't parse config file: yaml: line 117: mapping values are not allowed in this context 2021/07/10 10:38:45.731831 [fatal] yaml: line 117: mapping values are not allowed in this context [AdGuard-Home-Setup] Configuration error? Aborting...

seud0nym commented 3 years ago

It isn't bricked. Because DHCP is down, you will need to set a static IP on the connection to access the router.

Can you post up the lines around line 117 in the AdGuardHome.yaml file (it will be in the AdGuardHome directory on the USB).

seud0nym commented 3 years ago

To get everything back to working, run this:

curl -skL https://raw.githubusercontent.com/seud0nym/tch-gui-unhide/master/adguard/agh-setup | sh -s -- -r

jmytch commented 3 years ago

I will try it tonight when no one is using the internet lol

seud0nym commented 3 years ago

Thanks for the yaml file. I have deleted your post as requested.

Can you check your IPv6 LAN prefix for me? It will be shown on the Local Network Card or the Local Network screen. It will end in (probably) /64. In the yaml file it is shown as ending with :a000:: Is the prefix :a000::/64 or is there another digit (probably 1) before the /?

jmytch commented 3 years ago

24xx:xxxx:xxxx:A000::/64

That IPv6 LAN prefix is also my ipv6 ip address.

Change IPv6 Address Current IPv6 address 24xx:xxxx:xxxx:A000::/56 Next available IPv6 address 24xx:xxxx:xxxx:4500::/56

From Exetel member portal, so some how it automatically uses that with /64 for the LAN

seud0nym commented 3 years ago

Okay, so the bug is that I have to put the first address in the prefix range into the configuration, not the prefix range itself.

I have pushed up the change, so running the install again will pick up the fix.

I have also fixed the bug where it did not restart dnsmasq on install failure. Sorry about that.

jmytch commented 3 years ago

All good, will try again later tonight. Thanks for fix with the dnsmasq in case :)

jmytch commented 3 years ago

Just making sure everything is good to go? Here is some command I put in PuTTY I found from Whirlpool.

root@mymodem:~# df -h

Filesystem Size Used Available Use% Mounted on /dev/root 39.3M 39.3M 0 100% /rom tmpfs 339.8M 300.0K 339.6M 0% /tmp /dev/mtdblock2 89.1M 6.1M 83.0M 7% /overlay overlayfs:/overlay 89.1M 6.1M 83.0M 7% / tmpfs 512.0K 0 512.0K 0% /dev /dev/sda 29.8G 64.0K 29.8G 0% /tmp/run/mountd/sda

root@mymodem:~# cat /proc/mounts

/dev/root /rom squashfs ro,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,noatime 0 0 sysfs /sys sysfs rw,nosuid,nodev,noexec,noatime 0 0 tmpfs /tmp tmpfs rw,nosuid,nodev,noatime 0 0 /dev/mtdblock2 /overlay jffs2 rw,relatime,compr=zlib 0 0 overlayfs:/overlay / overlay rw,noatime,lowerdir=/,upperdir=/overlay/bank_2,workdir=/overlay/work 0 0 tmpfs /dev tmpfs rw,nosuid,relatime,size=512k,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,mode=600,ptmxmode=000 0 0 cgroup /cgroups/cpumemblk cgroup rw,relatime,cpuset,cpu,blkio,memory,clone_children 0 0 debugfs /sys/kernel/debug debugfs rw,noatime 0 0 mountd(pid7797) /tmp/run/mountd autofs rw,relatime,fd=5,pgrp=7796,timeout=600,minproto=5,maxproto=5,indirect 0 0 /dev/sda /tmp/run/mountd/sda vfat rw,noatime,fmask=0000,dmask=0000,allow_utime=0022,codepage=437,iocharset=utf8,shortname=mixed,errors=continue 0 0

root@mymodem:~# ll /mnt/usb/USB-A < mine does not contain any number after the letter

lrwxrwxrwx 1 root root 19 Jul 10 22:26 /mnt/usb/USB-A -> /tmp/run/mountd/sda/

So my USB says as /USB-A where other people gets /USB-A1 that contains a number after the A where mine doesn't. Would it affect me in a way if the script can read any mount USB?

Capture From the Content Sharing card.

jmytch commented 3 years ago

Everything is resolved, ran it tonight and it works beautifully. Thank you so much for adding this, and will as well close this :)

jmytch commented 3 years ago

Sorry, just overnight for some reason the router now show 23 MB of RAM left, I don't know what is going on with the RAM thing. And it is very close to be running out of RAM where it would restart the modem itself.

Any fix, tweak I should do? I did ./de-telstra -M -G and it still the same result. I think this firmware on 20.3c is using alot of RAM compared to before on 18.1 where it was much lower that has 550 MB RAM left.

I had to restart my modem and now its showing 180 MB RAM left which is good at the time adguard home was installed.

seud0nym commented 3 years ago

Did you define any extra block lists?

I am just trying to do some load testing on AdGuard Home, to see if I can work out what we can do to make it work better.

In the meantime, can you try turning off query logging? It is in the AdGuard Home General settings option under Settings. Make sure Enable log is unticked, and save.

Let me know if it makes any difference.

jmytch commented 3 years ago

Did you define any extra block lists?

Oh yeah I did use custom block list that has 380,305 domains blocked for tracking & ads from https://www.github.developerdan.com/hosts/lists/ads-and-tracking-extended.txt

can you try turning off query logging?

I could try that

jmytch commented 3 years ago

It is slowing down, but I might just revert back to 18.1 firmware along with AdGuard Home might be a better option since there will be plentiful of RAM left - around 550 MB free. I think 20.3c is too little on RAM for my liking. Telstra had done something to use up alot of RAM in the latest release.

seud0nym commented 3 years ago

Please re-open if you have any issues.

seud0nym commented 3 years ago

FYI: I have put up an extra script that adds support to the GUI for the OpenWRT adblock package. https://github.com/seud0nym/tch-gui-unhide/tree/master/extras#tch-gui-unhide-xtraadblock

seud0nym / tch-gui-unhide

Adblocking on DJA0230 #30