search

centminmod / centminmod-ultimate-bad-bot-blocker

Ultimate Bad Bot Blocker are for Centmin Mod 123.09beta01
https://centminmod.com/
12 stars 6 forks source link

UPDATES NEVER WORKED #3

Open mrEckendonk opened 4 years ago

mrEckendonk commented 4 years ago

This is the only working setup for https://github.com/centminmod/centminmod step-1

download and install wget https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/install-ngxblocker -O /usr/local/sbin/install-ngxblocker chmod +x /usr/local/sbin/install-ngxblocker

mkdir -p /usr/local/nginx/conf/ultimate-badbot-blocker

backup nginx.conf and conf.d directory before install cp -a /usr/local/nginx/conf/nginx.conf /usr/local/nginx/conf/nginx.conf-backup-b4-badbot cp -a /usr/local/nginx/conf/conf.d/ /usr/local/nginx/conf/conf.d-backup-b4-badbot

dry run install-ngxblocker -c /usr/local/nginx/conf/ultimate-badbot-blocker -b /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d

Checking url: https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/include_filelist.txt

Dry Run | not updating files | run as 'install-ngxblocker -x' to install files.

Creating directory: /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d

REPO = https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master

Downloading [FROM]=> [REPO]/conf.d/globalblacklist.conf [TO]=> /usr/local/nginx/conf/ultimate-badbot-blocker/globalblacklist.conf Downloading [FROM]=> [REPO]/conf.d/botblocker-nginx-settings.conf [TO]=> /usr/local/nginx/conf/ultimate-badbot-blocker/botblocker-nginx-settings.conf

REPO = https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master

Downloading [FROM]=> [REPO]/bots.d/blockbots.conf [TO]=> /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/blockbots.conf Downloading [FROM]=> [REPO]/bots.d/ddos.conf [TO]=> /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/ddos.conf Downloading [FROM]=> [REPO]/bots.d/custom-bad-referrers.conf [TO]=> /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/custom-bad-referrers.conf Downloading [FROM]=> [REPO]/bots.d/bad-referrer-words.conf [TO]=> /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/bad-referrer-words.conf Downloading [FROM]=> [REPO]/bots.d/blacklist-ips.conf [TO]=> /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/blacklist-ips.conf Downloading [FROM]=> [REPO]/bots.d/blacklist-user-agents.conf [TO]=> /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/blacklist-user-agents.conf Downloading [FROM]=> [REPO]/bots.d/whitelist-domains.conf [TO]=> /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/whitelist-domains.conf Downloading [FROM]=> [REPO]/bots.d/whitelist-ips.conf [TO]=> /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/whitelist-ips.conf

REPO = https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master

Downloading [FROM]=> [REPO]/setup-ngxblocker [TO]=> /usr/local/sbin/setup-ngxblocker Downloading [FROM]=> [REPO]/update-ngxblocker [TO]=> /usr/local/sbin/update-ngxblocker

cd /usr/local/sbin/

live run install-ngxblocker -x -c /usr/local/nginx/conf/ultimate-badbot-blocker -b /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d

install-ngxblocker -x -c /usr/local/nginx/conf/ultimate-badbot-blocker -b /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d Checking url: https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/include_filelist.txt

Creating directory: /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d

REPO = https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master

Downloading [FROM]=> [REPO]/conf.d/globalblacklist.conf [TO]=> /usr/local/nginx/conf/ultimate-badbot-blocker/globalblacklist.conf...OK Downloading [FROM]=> [REPO]/conf.d/botblocker-nginx-settings.conf [TO]=> /usr/local/nginx/conf/ultimate-badbot-blocker/botblocker-nginx-settings.conf...OK

REPO = https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master

Downloading [FROM]=> [REPO]/bots.d/blockbots.conf [TO]=> /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/blockbots.conf...OK Downloading [FROM]=> [REPO]/bots.d/ddos.conf [TO]=> /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/ddos.conf...OK Downloading [FROM]=> [REPO]/bots.d/custom-bad-referrers.conf [TO]=> /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/custom-bad-referrers.conf...OK Downloading [FROM]=> [REPO]/bots.d/bad-referrer-words.conf [TO]=> /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/bad-referrer-words.conf...OK Downloading [FROM]=> [REPO]/bots.d/blacklist-ips.conf [TO]=> /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/blacklist-ips.conf...OK Downloading [FROM]=> [REPO]/bots.d/blacklist-user-agents.conf [TO]=> /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/blacklist-user-agents.conf...OK Downloading [FROM]=> [REPO]/bots.d/whitelist-domains.conf [TO]=> /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/whitelist-domains.conf...OK Downloading [FROM]=> [REPO]/bots.d/whitelist-ips.conf [TO]=> /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/whitelist-ips.conf...OK

REPO = https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master

Downloading [FROM]=> [REPO]/setup-ngxblocker [TO]=> /usr/local/sbin/setup-ngxblocker...OK Downloading [FROM]=> [REPO]/update-ngxblocker [TO]=> /usr/local/sbin/update-ngxblocker...OK Setting mode: 700 => /usr/local/sbin/install-ngxblocker Setting mode: 700 => /usr/local/sbin/setup-ngxblocker Setting mode: 700 => /usr/local/sbin/update-ngxblocker

sudo chmod +x /usr/local/sbin/setup-ngxblocker sudo chmod +x /usr/local/sbin/update-ngxblocker

This is only a DRY-RUN so no changes are being made yet.

setup-ngxblocker -e conf -c /usr/local/nginx/conf/ultimate-badbot-blocker -b /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d -v /usr/local/nginx/conf/conf.d -m /usr/local/nginx/conf/nginx.conf Checking url: https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/include_filelist.txt

Dry Run | not updating files | run as 'setup-ngxblocker -x' to setup files.

inserting: include /usr/local/nginx/conf/ultimate-badbot-blocker/globalblacklist.conf; => /usr/local/nginx/conf/nginx.conf inserting: include /usr/local/nginx/conf/ultimate-badbot-blocker/botblocker-nginx-settings.conf; => /usr/local/nginx/conf/nginx.conf inserting: include /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/blockbots.conf; => /usr/local/nginx/conf/conf.d/demodomain.com.confr.ssl.conf inserting: include /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/ddos.conf; => /usr/local/nginx/conf/conf.d/demodomain.com.conf.ssl.conf inserting: include /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/blockbots.conf; => /usr/local/nginx/conf/conf.d/demodomain.com.conf inserting: include /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/ddos.conf; => /usr/local/nginx/conf/conf.d/demodomain.com.conf inserting: include /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/blockbots.conf; => /usr/local/nginx/conf/conf.d/phpmyadmin_ssl.conf inserting: include /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/ddos.conf; => /usr/local/nginx/conf/conf.d/phpmyadmin_ssl.conf inserting: include /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/blockbots.conf; => /usr/local/nginx/conf/conf.d/virtual.conf inserting: include /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/ddos.conf; => /usr/local/nginx/conf/conf.d/virtual.conf setup will fix conflict from: 'server_names_hash_bucket_size' in /usr/local/nginx/conf/ultimate-badbot-blocker/botblocker-nginx-settings.conf setup will fix conflict from: 'server_names_hash_max_size' in /usr/local/nginx/conf/ultimate-badbot-blocker/botblocker-nginx-settings.conf setup will fix conflict from: 'limit_req_zone' in /usr/local/nginx/conf/ultimate-badbot-blocker/botblocker-nginx-settings.conf

Whitelisting ip: x.x.x.x: => /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/whitelist-ips.conf

Web directory not found ('/var/www'): not automatically whitelisting domains.

Checking for missing includes:

Checking url: https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/include_filelist.txt

Nothing to update for directory: /usr/local/nginx/conf/ultimate-badbot-blocker Nothing to update for directory: /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d Nothing to update for directory: /usr/local/sbin Setting mode: 700 => /usr/local/sbin/install-ngxblocker Setting mode: 700 => /usr/local/sbin/setup-ngxblocker Setting mode: 700 => /usr/local/sbin/update-ngxblocker Updating bots.d path: /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d => /usr/local/nginx/conf/ultimate-badbot-blocker/globalblacklist.conf

setup-ngxblocker -x -e conf -c /usr/local/nginx/conf/ultimate-badbot-blocker -b /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d -v /usr/local/nginx/conf/conf.d -m /usr/local/nginx/conf/nginx.conf Checking url: https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/include_filelist.txt

inserting: include /usr/local/nginx/conf/ultimate-badbot-blocker/globalblacklist.conf; => /usr/local/nginx/conf/nginx.conf inserting: include /usr/local/nginx/conf/ultimate-badbot-blocker/botblocker-nginx-settings.conf; => /usr/local/nginx/conf/nginx.conf inserting: include /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/blockbots.conf; => /usr/local/nginx/conf/conf.d/demodomain.com.conf.ssl.conf inserting: include /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/ddos.conf; => /usr/local/nginx/conf/conf.d/demodomain.com.conf.ssl.conf inserting: include /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/blockbots.conf; => /usr/local/nginx/conf/conf.d/demodomain.com.conf inserting: include /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/ddos.conf; => /usr/local/nginx/conf/conf.d/demodomain.com.conf inserting: include /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/blockbots.conf; => /usr/local/nginx/conf/conf.d/phpmyadmin_ssl.conf inserting: include /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/ddos.conf; => /usr/local/nginx/conf/conf.d/phpmyadmin_ssl.conf inserting: include /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/blockbots.conf; => /usr/local/nginx/conf/conf.d/virtual.conf inserting: include /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/ddos.conf; => /usr/local/nginx/conf/conf.d/virtual.conf setup will fix conflict from: 'server_names_hash_bucket_size' in /usr/local/nginx/conf/ultimate-badbot-blocker/botblocker-nginx-settings.conf disabling 'server_names_hash_bucket_size' in: /usr/local/nginx/conf/ultimate-badbot-blocker/botblocker-nginx-settings.conf disabled OK

setup will fix conflict from: 'server_names_hash_max_size' in /usr/local/nginx/conf/ultimate-badbot-blocker/botblocker-nginx-settings.conf disabling 'server_names_hash_max_size' in: /usr/local/nginx/conf/ultimate-badbot-blocker/botblocker-nginx-settings.conf disabled OK

setup will fix conflict from: 'limit_req_zone' in /usr/local/nginx/conf/ultimate-badbot-blocker/botblocker-nginx-settings.conf disabling 'limit_req_zone' in: /usr/local/nginx/conf/ultimate-badbot-blocker/botblocker-nginx-settings.conf disabled OK

Whitelisting ip: x.x.x.x:: => /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d/whitelist-ips.conf

Web directory not found ('/var/www'): not automatically whitelisting domains.

Checking for missing includes:

Checking url: https://raw.githubusercontent.com/mitchellkrogza/nginx-ultimate-bad-bot-blocker/master/include_filelist.txt

Nothing to update for directory: /usr/local/nginx/conf/ultimate-badbot-blocker Nothing to update for directory: /usr/local/nginx/conf/ultimate-badbot-blocker/bots.d Nothing to update for directory: /usr/local/sbin Setting mode: 700 => /usr/local/sbin/install-ngxblocker Setting mode: 700 => /usr/local/sbin/setup-ngxblocker Setting mode: 700 => /usr/local/sbin/update-ngxblocker

nginx.conf fix

nano /usr/local/nginx/conf/ultimate-badbot-blocker/botblocker-nginx-settings.conf

server_names_hash_bucket_size 256;

server_names_hash_max_size 4096;

variables_hash_max_size 4096;

variables_hash_bucket_size 4096;

limit_req_zone $binary_remote_addr zone=flood:50m rate=90r/s;

limit_req_zone $binary_remote_addr zone=flood:10m rate=30r/s;

edit /usr/bin/sbin/update/nginxblocker

CONF_DIR=/etc/nginx/conf.d

CONF_DIR=/usr/local/nginx/conf/ultimate-badbot-blocker/conf.d

BOTS_DIR=/etc/nginx/bots.d

BOTS_DIR=/usr/local/nginx/conf/ultimate-badbot-blocker/bots.d

step-10

DISABLE CLOUDFLARE TEST THAT IT IS WORKING)

TESTING

Run the following commands one by one from a terminal on another linux machine against your own domain name.

❗️ substitute http:// yourdomain.com ❗️ in the examples below with your own REAL domain name ❗️

curl -A "Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.96 Mobile Safari/537.36 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)" -I http://yourdomain.com

curl -A "Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)" -I http://yourdomain.com

Should respond with 200 OK

curl -A "Xenu Link Sleuth/1.3.8" -I http://yourdomain.com

curl -A "Mozilla/5.0 (compatible; AhrefsBot/5.2; +http://ahrefs.com/robot/)" -I http://yourdomain.com

Should respond with either of the following error messages:

curl: (52) Empty reply from server curl: (56) TCP connection reset by peer curl: (92) HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1) curl -I http://yourdomain.com -e http://100dollars-seo.com

curl -I http://yourdomain.com -e http://zx6.ru

Should respond with either of the following error messages:

curl: (52) Empty reply from server curl: (56) TCP connection reset by peer curl: (92) HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1) The Nginx Ultimate Bot Blocker is now WORKING and PROTECTING your web sites !!!

❗️ substitute http:// yourdomain.com ❗️ in the examples below with your own REAL domain name ❗️

ENABLE CLOUDFLARE

centminmod commented 4 years ago

updates for what ? it isn't clear what isn't working ?

mrEckendonk commented 4 years ago

If you don't change this

edit /usr/bin/sbin/update/nginxblocker
#CONF_DIR=/etc/nginx/conf.d
CONF_DIR=/usr/local/nginx/conf/ultimate-badbot-blocker/conf.d
#BOTS_DIR=/etc/nginx/bots.d
BOTS_DIR=/usr/local/nginx/conf/ultimate-badbot-blocker/bots.d

In the file /usr/bin/sbin/update/nginxblocker it creates the updated files in /etc/nginx/bots.d, so they never ever get updated.

Sent in some bots, Mitchell updated it, but it kept coming back, so scratching my head, WTF is wrong.

You receive a mail that it is updated fine, but it ain't.

centminmod commented 4 years ago

I see that would be issue on Mitchell's end

mrEckendonk commented 4 years ago

Play ping pong with Mitchell. Ball bounce forward and back.

#CONF_DIR=/etc/nginx/conf.d
CONF_DIR=/usr/local/nginx/conf/ultimate-badbot-blocker/conf.d
#BOTS_DIR=/etc/nginx/bots.d
BOTS_DIR=/usr/local/nginx/conf/ultimate-badbot-blocker/bots.d

A normal Centos7 has its nginx config on the default place:

Default NGINX Server Root The default NGINX server root directory is /usr/share/nginx. This is specified in the default server block configuration file, located at /etc/nginx/conf.d/default.conf.

The default server document root directory which contains web files is usr/share/nginx/html.

Global Configuration Global configurations can be tweaked by modifying the main NGINX configuration file located at /etc/nginx/nginx.conf. This configuration file is broken down into contexts.