anudeepND / whitelist

A simple tool to add commonly white listed domains to your Pi-Hole setup.
MIT License
4.17k stars 283 forks source link

Problem whith whitelist.py #265

Open osespe opened 10 months ago

osespe commented 10 months ago

When I run sudo python3 whitelist/scripts/whitelist.py, it adds the lists, but when it reaches the section "Restarting Pi-hole. This could take a few seconds," it just hangs without doing anything else. The only way is to do a Ctrl+C, and it gives the following error:

This script will download and add domains from the repo to whitelist. All the domains in this list are safe to add and does not contain any tracking or adserving domains.

[i] Pi-hole path exists [i] Write access to /etc/pihole verified [i] Pi-Hole Gravity database found [i] 191 domains and 191 SQL queries discovered [i] Connecting to Gravity. [i] Successfully Connected to Gravity. [i] Checking Gravity for domains added by script. [i] 96 Domains from script in whitelist already. [i] Checking Gravity for domains added by user that are also in script. [i] Found 0 domains added by the user that would be added by script. [i] Checking Gravity for domains previously added by script that are NOT in new script. [i] Found 0 domain(s) added previously by script that are not in script. [i] Checking script for domains not in Gravity. [i] 95 domain(s) NOT in Gravity that are in new script.

- Adding android.clients.google.com
- Adding appleid.apple.com
- Adding apps.skype.com
- Adding apt.sonarr.tv
- Adding attestation.xboxlive.com
- Adding c.s-microsoft.com
- Adding cdn.vidible.tv
- Adding cert.mgt.xboxlive.com
- Adding clientconfig.passport.net
- Adding clients2.google.com
- Adding clients3.google.com
- Adding clients4.google.com
- Adding cpms.spop10.ams.plex.bz
- Adding cpms35.spop10.ams.plex.bz
- Adding ctldl.windowsupdate.com
- Adding livepassdl.conviva.com
- Adding gfwsl.geforce.com
- Adding appspot-preview.l.google.com
- Adding dashboard.plex.tv
- Adding def-vef.xboxlive.com
- Adding delivery.vidible.tv
- Adding dev.virtualearth.net
- Adding device.auth.xboxlive.com
- Adding displaycatalog.mp.microsoft.com
- Adding dl.delivery.mp.microsoft.com
- Adding dl.dropboxusercontent.com
- Adding download.sonarr.tv
- Adding ecn.dev.virtualearth.net
- Adding edge.api.brightcove.com
- Adding eds.xboxlive.com
- Adding forums.sonarr.tv
- Adding g.live.com
- Adding geo-prod.do.dsp.mp.microsoft.com
- Adding gravatar.com
- Adding help.ui.xboxlive.com
- Adding imagesak.secureserver.net
- Adding img.vidible.tv
- Adding licensing.xboxlive.com
- Adding login.live.com
- Adding login.microsoftonline.com
- Adding meta-db-worker02.pop.ric.plex.bz
- Adding meta.plex.bz
- Adding meta.plex.tv
- Adding msftncsi.com
- Adding nexusrules.officeapps.live.com
- Adding nine.plugins.plexapp.com
- Adding node.plexapp.com
- Adding notify.xboxlive.com
- Adding ns1.dropbox.com
- Adding ns2.dropbox.com
- Adding o1.email.plex.tv
- Adding o2.sg0.plex.tv
- Adding ocsp.apple.com
- Adding officeclient.microsoft.com
- Adding outlook.office365.com
- Adding placehold.it
- Adding placeholdit.imgix.net
- Adding pricelist.skype.com
- Adding products.office.com
- Adding proxy.plex.bz
- Adding proxy.plex.tv
- Adding proxy02.pop.ord.plex.bz
- Adding pubsub.plex.bz
- Adding pubsub.plex.tv
- Adding s.gateway.messenger.live.com
- Adding s.marketwatch.com
- Adding s.youtube.com
- Adding sa.symcb.com
- Adding services.sonarr.tv
- Adding skyhook.sonarr.tv
- Adding spclient.wg.spotify.com
- Adding staging.plex.tv
- Adding status.plex.tv
- Adding t0.ssl.ak.dynamic.tiles.virtualearth.net
- Adding t0.ssl.ak.tiles.virtualearth.net
- Adding themoviedb.com
- Adding thetvdb.com
- Adding title.auth.xboxlive.com
- Adding title.mgt.xboxlive.com
- Adding tvdb2.plex.tv
- Adding tvthemes.plexapp.com
- Adding ui.skype.com
- Adding video-stats.l.google.com
- Adding videos.vidible.tv
- Adding widget-cdn.rpxnow.com
- Adding www.googleapis.com
- Adding www.msftncsi.com
- Adding xbox.ipv6.microsoft.com
- Adding xboxexperiencesprod.experimentation.xboxlive.com
- Adding xflight.xboxlive.com
- Adding xkms.xboxlive.com
- Adding xsts.auth.xboxlive.com
- Adding connectivitycheck.gstatic.com
- Adding connectivitycheck.android.com
- Adding www.msftconnecttest.com

[i] Checking Gravity for newly added domains.

[i] All 95 new domain(s) have not been added to Gravity. [i] There are a total of 566 domains in your whitelist (regex(13) & exact(553)) [i] The database connection is closed [i] Restarting Pi-hole. This could take a few seconds ^C

Done. Happy ad-blocking :)

Star me on GitHub: https://github.com/anudeepND/whitelist Buy me a coffee: https://paypal.me/anudeepND

Traceback (most recent call last): File "/home/x/whitelist/scripts/whitelist.py", line 336, in restart_pihole(args.docker) File "/home/x/whitelist/scripts/whitelist.py", line 60, in restart_pihole subprocess.call(['pihole', '-g'], stdout=subprocess.DEVNULL) File "/usr/lib/python3.10/subprocess.py", line 347, in call return p.wait(timeout=timeout) File "/usr/lib/python3.10/subprocess.py", line 1209, in wait return self._wait(timeout=timeout) File "/usr/lib/python3.10/subprocess.py", line 1959, in _wait (pid, sts) = self._try_wait(0) File "/usr/lib/python3.10/subprocess.py", line 1917, in _try_wait (pid, sts) = os.waitpid(self.pid, wait_flags) KeyboardInterrupt

Technetium1 commented 10 months ago

Please check https://github.com/anudeepND/whitelist/issues/264, does it help?

osespe commented 10 months ago

I think the problem has nothing to do with that incident. I don't use Docker; it's an installation on Ubuntu 22 without Docker. I've noticed that the errors in the logs are also different. Pi-hole is working.

Technetium1 commented 10 months ago

Oh no! I don't have pihole for now, so I can't test this further. I sure hope this isn't caused by a recent major change. If so, then the scripts used here may need updated accordingly.

osespe commented 10 months ago

Certainly, it's what you're saying. I installed Pi-hole this week; I've never had it before.

osespe commented 10 months ago

I tried with the root user, left it for an hour, and when I came back, it didn't give an error. I'm going to try with the other user.

osespe commented 10 months ago

Confirmed, it's working fine now. From the moment I execute the command until it finishes, it takes 11 minutes.

Technetium1 commented 10 months ago

@osespe can you please use root and report the outputs of which pihole, which docker, and printenv then run the 3 of those commands as the user which you tried to run sudo python3 whitelist/scripts/whitelist.py with originally? https://github.com/anudeepND/whitelist/blob/30c75868f56a4d46b3cf164e184cd888ee254fe2/scripts/whitelist.py#L55-L60

If Docker exists, it assumes Docker is running pihole. Here we should probably be checking to see if Docker exists, then if it exists, see if there is also a container called pihole, and NOT call it if the container doesn't exist.

osespe commented 10 months ago

I don't use Docker; I have Pi-hole installed without Docker, directly on Ubuntu.

On the other hand, randomly, whether with root or another user with permissions, I get this error:

This script will download and add domains from the repo to whitelist. All the domains in this list are safe to add and does not contain any tracking or adserving domains.

[i] Pi-hole path exists [i] Write access to /etc/pihole verified [X] URL Error: [SSL: UNEXPECTED_EOF_WHILE_READING] EOF occurred in violation of protocol (_ssl.c:1007) whilst fetching https://raw.githubusercontent.com/anudeepND/whitelist/master/domains/whitelist.txt

Usually, I run it again, and it works fine.