search

anudeepND / whitelist

A simple tool to add commonly white listed domains to your Pi-Hole setup.
MIT License
4.21k stars 281 forks source link

Gravity is never updated after script is run #180

Closed Dantasstic closed 3 years ago

Dantasstic commented 3 years ago

21 plex domains keep saying they are being added, but if I run the script again, it just says the same thing over and over:

[i] 21 domain(s) NOT in Gravity that are in new script.

- Adding cdn.optimizely.com
- Adding clients4.google.com
- Adding cpms.spop10.ams.plex.bz
- Adding cpms35.spop10.ams.plex.bz
- Adding meta-db-worker02.pop.ric.plex.bz
- Adding meta.plex.bz
- Adding meta.plex.tv
- Adding nine.plugins.plexapp.com
- Adding node.plexapp.com
- Adding o1.email.plex.tv
- Adding proxy.plex.bz
- Adding proxy.plex.tv
- Adding proxy02.pop.ord.plex.bz
- Adding pubsub.plex.bz
- Adding pubsub.plex.tv
- Adding spclient.wg.spotify.com
- Adding staging.plex.tv
- Adding status.plex.tv
- Adding tinyurl.com
- Adding tvdb2.plex.tv
- Adding tvthemes.plexapp.com

[i] Checking Gravity for newly added domains.

[i] All 21 new domain(s) have not been added to Gravity. [i] There are a total of 218 domains in your whitelist (regex(0) & exact(218)) [i] The database connection is closed [i] Restarting Pi-hole. This could take a few seconds

Done. Happy ad-blocking :)

mwoolweaver commented 3 years ago

@Dantasstic can you try the script found here and let me know how it works for you?

Dantasstic commented 3 years ago

How do I do that?

mwoolweaver commented 3 years ago 
curl -sSL https://raw.githubusercontent.com/mwoolweaver/whitelist/master/scripts/whitelist.py | sudo /usr/bin/env python3

make sure you get the whole line

Dantasstic commented 3 years ago

This script will download and add domains from the repo to whitelist. All the domains in this list are safe to add and does not contain any tracking or adserving domains.

[i] Pi-hole path exists [i] Write access to /etc/pihole verified [i] Pi-Hole Gravity database found [i] 191 domains and 191 SQL queries discovered [i] Connecting to Gravity. [i] Successfully Connected to Gravity. [i] Checking Gravity for domains added by script. [i] Found 170 domains added by script in whitelist already. [i] Checking Gravity for domains added by user that are also in script. [i] Checking Gravity for anudeepND/whitelist group.

[i] Checking Gravity for newly added domains.

[i] All 21 new domain(s) have not been added to Gravity. [i] All 191 domains to be added by script have been discovered in Gravity [i] There are a total of 218 domains in your whitelist (regex(0) & exact(218)) [i] The database connection is closed [i] Reloading Pi-hole lists. This could take a few seconds

[✓] Flushing DNS cache

Done. Happy ad-blocking :)

Star me on GitHub: https://github.com/anudeepND/whitelist Buy me a coffee: https://paypal.me/anudeepND

mwoolweaver commented 3 years ago

It gives you that output all every time you run it?

mwoolweaver commented 3 years ago

@Dantasstic would try again by running that same command again?

mwoolweaver commented 3 years ago

@Dantasstic if you try `pihole -q tvthemes.plexapp.com' and post the output? I think i figured out the issue

Dantasstic commented 3 years ago

@Dantasstic if you try `pihole -q tvthemes.plexapp.com' and post the output? I think i figured out the issue

pihole -q tvthemes.plexapp.com Match found in exact whitelist tvthemes.plexapp.com.cdn.cloudflare.net tvthemes.plexapp.com

mwoolweaver commented 3 years ago

and if you

  1. go to http://pi.hole/admin/groups-domains.php?type=white

  2. search for that same domain

does the entry have a comment?

mwoolweaver commented 3 years ago

@Dantasstic Try 

curl -sSL https://raw.githubusercontent.com/mwoolweaver/whitelist/master/scripts/whitelist.py | sudo /usr/bin/env python3

it should work correctly for you now :crossed_fingers:

Dantasstic commented 3 years ago

[i] Pi-hole path exists [i] Write access to /etc/pihole verified [i] Pi-Hole Gravity database found [i] 191 domains and 191 SQL queries discovered [i] Connecting to Gravity. [i] Successfully Connected to Gravity. [i] Checking Gravity for domains added by script. [i] Found 170 domains added by script in whitelist already. [i] Checking Gravity for domains added by user that are also in script. [i] Checking Gravity for anudeepND/whitelist group. [i] Found anudeepND/whitelist group in Gravity. [i] 21 domain(s) added by the user that would be added by script.

1. tinyurl.com
2. cdn.optimizely.com
3. clients4.google.com
4. spclient.wg.spotify.com
5. tvdb2.plex.tv
6. pubsub.plex.bz
7. proxy.plex.bz
8. proxy02.pop.ord.plex.bz
9. cpms.spop10.ams.plex.bz
10. meta-db-worker02.pop.ric.plex.bz
11. meta.plex.bz
12. tvthemes.plexapp.com
13. meta.plex.tv
14. cpms35.spop10.ams.plex.bz
15. proxy.plex.tv
16. pubsub.plex.tv
17. status.plex.tv
18. node.plexapp.com
19. nine.plugins.plexapp.com
20. staging.plex.tv
21. o1.email.plex.tv

[i] Checking Gravity for domains previously added by script that are NOT in new script. [i] Found no domain(s) added previously by script that are not in script. [i] Checking script for domains not in Gravity. [i] All 191 domains to be added by script have been discovered in Gravity [i] There are a total of 218 domains in your whitelist (regex(0) & exact(218)) [i] The database connection is closed

mwoolweaver commented 3 years ago

@Dantasstic So basically what was happening is those domains didn't have a comment and my sql statement was missing them because they return null so when checking for the absence of the string the script leaves as a comment it wasn't finding any text at all so it would skip it.

Now I'm checking for both the absence of our comment and a null value.

[i] 21 domain(s) added by the user that would be added by script.

means you have manually added these domains so the script will not mess with them.

[i] All 191 domains to be added by script have been discovered in Gravity.

Means all 191 domains the script has in it have been found in Gravity.

So you should be all good now. Thanks for testing these for me so I could fix the issue.

Dantasstic commented 3 years ago

Thanks for the help!

So going forward I should use this command to check for whitelist updates:

@Dantasstic Try

curl -sSL https://raw.githubusercontent.com/mwoolweaver/whitelist/master/scripts/whitelist.py | sudo /usr/bin/env python3

it should work correctly for you now 🤞

mwoolweaver commented 3 years ago

only until #184 gets merged