mikelynn2 / blacklistmonitor

Web based application for monitoring Domains and IPs on RBLs
MIT License
85 stars 42 forks source link

Does this work? How to confirm? #3

Closed amityweb closed 9 years ago

amityweb commented 9 years ago

I installed this software because I know one of our servers is on SPAMCOP and SORBS. MXToolbox free account told me. But this software reports all my servers are OK. So is this software working/accurate? How can I confirm?

Thanks

amityweb commented 9 years ago

Would it be due to not running BIND on the server, and using 127.0.0.1 as the DNS server? I'd get errors if could not reach the hostnames to check surely? Thanks

mikelynn2 commented 9 years ago

thats right. you need a bind server. And you can't use public DNS servers like google or opendns because many RBLs block them. You most likely need to run your own to get good results.

amityweb commented 9 years ago

Yes I tried Google first! :)

I will install BIND and see if that works.

Thanks

amityweb commented 9 years ago

I'm getting into unknown territory here so will have to do more research. Because installing and using BIND (so DNS is 127.0.0.1) results in unknown hosts from my server, no domain name can be resolved. I hoped adding a forwarder in BIND will work, so hosts are now resolved, but then they are blocked due to being seen as coming from Google again. So I guess there is more to setting up BIND than I thought.

mikelynn2 commented 9 years ago

You don't want a forwarder. That would just cache and proxy requests to google basicly. You want caching only. You should be very close to that now. Like this : http://www.tecmint.com/install-caching-only-dns-server-in-centos/ What centos version?

When this resolves you know you got it right. Or whatever IP bind is listening on. dig @127.0.0.1 github.com

mikelynn2 commented 9 years ago

the service script isn't going to work for you either on CentOS. I'll need to write one for that

amityweb commented 9 years ago

Oh, it wont work on CentOS? Oh. Centos 6.6. I didnt realise it wont work on Ubuntu. May aswell wait until it does before doing al the BIND stuff. Maybe add in some additional checking, because my hosts all report OK, no errors anywhere, yet we know now its not actually working. Thanks

mikelynn2 commented 9 years ago

Well I didn't think the upstart would but if your showing OKs then it probably did run. Do you know if any hosts are listed? It shouldn't show OK unless they've been checked

On Jun 12, 2015, at 3:25 AM, Laurence Cope notifications@github.com wrote:

Oh, it wont work on CentOS? Oh. Centos 6.6. I didnt realise it wont work on Ubuntu. May aswell wait until it does before doing al the BIND stuff. Maybe add in some additional checking, because my hosts all report OK, no errors anywhere, yet we know now its not actually working. Thanks

— Reply to this email directly or view it on GitHub.

amityweb commented 9 years ago

yes all hosts are listed. They all showed OK when using 127.0.0.1 but without BIND running. Then if I use Googles DNS it showed it was blocked. So looks like the process is working, just need to setup BIND by the looks of it.

As a sidenote, how come we need to go through DNS servers? On this server I have a PHP script to check websites and I just use CURL to check them direct, do the blacklist services APIs or however they work just not accept a direct connection from our server to check the hostname? Dont understand why we are going through a DNS server.

mikelynn2 commented 9 years ago

without bind running the queries are timing out, so erroring on the side of being OK. The DNS server is going out to the blacklist servers API (their dns servers) so in order to talk to their DNS server you want to use a local DNS server. You could query them directly but using a DNS server in the middle is more flexable and it caches the requests so you don't hammer the blacklist servers DNS and get banned. your php script is using the dns servers in /etc/resolve.conf

amityweb commented 9 years ago

Thanks for your help on this, but I might give up :(. I setup BIND to cache but still get the errors. If I do a dig on a hostname, I can see its looking up from 127.0.0.1, and subsequent lookups are 0 seconds, so its using cache, but in the blacklist monitor I still get the errors: multi.uribl.com - myhostname.co.uk.multi.uribl.com. 1903 IN TXT 127.0.0.1 -> Query Refused. See http://uribl.com/refused.shtml for more information [Your DNS IP: 74.125.181.85]

Its a bit strange because I cant see anywhere where its set to use Google's DNS servers anymore. ifcfg-eth0 is set to use 127.0.0.1.

Blacklist monitor is set to use 127.0.0.1 in the config file.

If I enter the IP addresses direct, then looks like thats working, but ideally I would like to use hostnames so I can easily identify what servers they are.

Maybe its an issue for me to ask Digital Ocean as opposed to your system.

mikelynn2 commented 9 years ago

That's Google DNS servers. Somehow it's using them still.

http://whois.domaintools.com/74.125.181.85

Your local DNS server is probably forwarding to google

On Jun 13, 2015, at 8:51 AM, Laurence Cope notifications@github.com wrote:

Thanks for your help on this, but I might give up :(. I setup BIND to cache but still get the errors. If I do a dig on a hostname, I can see its looking up from 127.0.0.1, and subsequent lookups are 0 seconds, so its using cache, but in the blacklist monitor I still get the errors: multi.uribl.com - myhostname.co.uk.multi.uribl.com. 1903 IN TXT 127.0.0.1 -> Query Refused. See http://uribl.com/refused.shtml for more information [Your DNS IP: 74.125.181.85]

Blacklist monitor is set to use 127.0.0.1 in the config file.

If I enter the IP addresses direct, then looks like thats working, but ideally I would like to use hostnames so I can easily identify what servers they are.

— Reply to this email directly or view it on GitHub.

amityweb commented 9 years ago

I've raised it with Digital Ocean as I cant find any reference to using Google! I know Digital Ocean use Google DNS when you create a new Droplet, but I changed that, maybe something left over. If I figure it out I will update here for others.

mikelynn2 commented 9 years ago

can you show me your /etc/named.conf

amityweb commented 9 years ago

Ah, I may have just figured it out, I had the forwarders left over in there from when I tried it yesterday. Removed that and seems to work OK! I have a couple of known spam IP addresses, and they are showing as blocked, and then my servers are all OK. My recent blacklist was removed earlier so can't test my own server.

Thanks a lot for your help!! Good system.

amityweb commented 9 years ago

Actually I am still not sure its working. If I replace the IP address of the known spammer with its hostname (1-160-44-67.dynamic.hinet.net) which is on spamlists, it states its OK. So it seems only IP addresses work for me.

mikelynn2 commented 9 years ago

It's not going to check the IP of a host. Only the IPs and hosts you enter directly. So it won't check the IP associated to that host only the IP.

On Jun 13, 2015, at 3:01 PM, Laurence Cope notifications@github.com wrote:

Actually I am still not sure its working. If I replace the IP address of the known spammer with its hostname (1-160-44-67.dynamic.hinet.net) which is on spamlists, it states its OK. So it seems only IP addresses work for me.

— Reply to this email directly or view it on GitHub.

amityweb commented 9 years ago

Don't quite understand what you mean. I assume in the hosts box, if I enter my server hostname, it would check that the hostname IP is on a blacklist or not. If I enter an IP in the IP box then it would just check the IP (In fact entering an IP in the hosts box works too).

When I get more time soon, I will try adding all my servers IP addresses and see if it works. It works for the IP for 1-160-44-67.dynamic.hinet.net (bit not this domain name).

amityweb commented 9 years ago

To confirm, its working with IP addresses. So I can use IP addresses, and not hostnames. The system shows the Rdns hostname in the dashboard anyway, so I can see what servers they are for.

I guess you meant above that it checks if the hostname is listed, and does not get the IP of the hostname and then check that? That explains why it was all reporting OK. Seems safer to stick with IPs.

Thanks