open-source-ideas / ideas

💡 Looking for inspiration for your next open source project? Or perhaps you've got a brilliant idea you can't wait to share with others? Open Source Ideas is a community built specifically for this! 👋
6.59k stars 220 forks source link

Complaint Reporting Tool for Abuse/Hacking | Django #113

Open glenjarvis opened 6 years ago

glenjarvis commented 6 years ago

Project description

If one were to create a new basic webserver, listening on :80, it would not take long for hacker attempts to be made against that site. For example, one may see an HTTP GET Request for /.bitcoin/wallet.dat or /admin/config.php or /user/register?element_parents=account/mail/%23value&ajax_form=1&_wrapper_format=drupal_ajax. These are clear attempts to access the website in a non-authorized way. This is especially true if the site is not BitCoin mining related, a PHP website or a Drupal site.

Although there is little harm for these types of Kiddie Scripting attempts, they often can be annoying for a website using Django technology.

In Django on an AWS node behind a load balancer, the HTTP_X_FORWARDED_FOR header contain a list of IP addresses. The first is the source IP, for example: 2.203.36.157.

Thus, we can know that this IP address (randomly chosen) 2.203.36.157 has made an access request to one of the URLs in the table below. By Reverse IP Lookup, we can determine the source of the original sender. In this particular case it is: Vodophone DSL in Solingen, North Rhine-Westphalia, Germany, Europe.

Often there are multiple simultaneous requests from users that hit a URL from the table below. If there are many attempts (for example, if all three of these URLs were accessed) /.bitcoin/wallet.dat and /admin/config.php and /user/register?element_parents=account/mail/%23value&ajax_form=1&_wrapper_format=drupal_ajax we can safely assume this is a non-authorized attempt (especially if the site is a Django site).

This project is a Django Middleware that:

  1. Efficiently checks incoming IP address against a black list of IP addresses. Return 403 if on list.
  2. Efficiently checks URLs against the URLs in Table 1.1 below
  3. All hits are registered in a configurable backend with IP address and URL from Table 1.1. This backend can be efficient (such as a Redis node) or Simple (such as a database table).
  4. If there are more than a BAD_URL_NUM_THRESHOLD (or similar configurable variable), then this IP address is listed in the efficient list for returning 403s as shown in step 1 above

There are additional manage.py commands that can be run on a regular basis to:

  1. Write out the 403 Black List (from step 1 of previous MIddleware) in either Nginx or Apache Deny format so that this can be 403'd on the Web Server level in the future
  2. Collate an "Abuse Report" from this IP address. The format should include questions such as the following: https://aws.amazon.com/forms/report-abuse. The purpose of this report is to be a standard for submitting to Amazon - including logs and times needed to report the abuse. One may consider an option for Reverse Lookup of IP address (such as https://www.maxmind.com/en/geoip-demo.). A database of abuse@.. email addresses for the report would be very handy (and may be its own project)
  3. Alert when new abuse reports are available (from steps 1 and 2 above)

Often we don't complain because the information is too hard to collate in order to make the report. And, there are some companies, such as Amazon and Google who would respond to such complaints.

One should always be able to choose to manually review any reports before they are submitted. There may be some reports that we do not wish to send.

Table 1.1: Sample unauthorized URL patterns that I have seen against a Django site:

/.bitcoin/wallet.dat
/.env
/.ftpconfig
/.git/HEAD
/.git/index
/.idea/WebServers.xml
/.remote-sync.json
/.ssh/id_dsa
/.ssh/id_ecdsa
/.ssh/id_ed25519
/.ssh/id_rsa
/.svn/entries
/.vscode/ftp-sync.json
/.vscode/sftp.json
/.well-known/assetlinks.json
/.well-known/security.txt
//a2billing/customer/templates/default/footer.tpl
//admin/config.php
//blog/
//blog/wp-includes/wlwmanifest.xml
//cms/wp-includes/wlwmanifest.xml
//connectors/system/phpthumb.php
//dbs.php
//freepbx/config.php
//libs/js/iframe.js
//site/wp-includes/wlwmanifest.xml
//wordpress/wp-includes/wlwmanifest.xml
//wp-includes/wlwmanifest.xml
//wp-login.php
//wp/wp-includes/wlwmanifest.xml
//xmlrpc.php
/2phpmyadmin/
/404.jpg
/?XDEBUG_SESSION_START=phpstorm
/API/DW/Dwplugin/SystemLabel/SiteConfig.htm
/API/DW/Dwplugin/TemplateManage/login_site.htm
/API/DW/Dwplugin/TemplateManage/manage_site.htm
/API/DW/Dwplugin/TemplateManage/save_template.htm
/Admin/Common/HelpLinks.xml
/Admin/Images/LoginImages/admin_text.gif
/Admin/Images/LoginImages/admin_top.gif
/Admin/Login.aspx
/AdminManager/AdminLogo.aspx
/AdminManager/Images/adminLogin.gif
/AdminManager/images/top.gif
/App_Themes/Login/blue/css/login.css
/Article/Js/test.Js
/Childsplay.wmv
/CmxAbout.php
/CmxSupport.php
/Config/ZL_License.txt
/Conn.asp
/Count/CounterLink.asp
/CuteSoft_Client/CuteEditor/Help/default.htm
/CuteSoft_Client/CuteEditor/ImageEditor/listfiles.aspx
/CuteSoft_Client/CuteEditor/Images/log.gif
/CuteSoft_Client/CuteEditor/Style/IE.css
/Editor.js
/Error.aspx
/FCK/editor/js/fckeditorcode_ie.js
/FCK/fckeditor.js
/FileZilla.xml
/Forum/
/Frm/login.aspx
/HNAP1
/HNAP1/
/Images/Img1/loginbg.jpg
/Images/log2.jpg
/Images/login/biaoti.jpg
/Images/login/lefttu.jpg
/Images/login/mainlogo.gif
/Images/logo.png
/License.txt
/Login.jsp
/Ntalker/lawfirm.aspx
/PMA/
/PMA2011/
/PMA2012/
/PMA2013/
/PMA2014/
/PMA2015/
/PMA2016/
/PMA2017/
/PMA2018/
/Prompt/images/P_Wrong.gif
/README.txt
/Resource/Counter.aspx
/Scripts/ckeditor/ckeditor.js
/Scripts/jquery/maticsoft.jquery.min.js
/Search.html
/ServerInfo.txt
/System/sys_login_eos.asp
/Template/Default/Skin/user/images/login_back.jpg
/User/Login.aspx
/UserCenter/css/admin/bgimg/admin_all_bg.png
/WS_FTP.INI
/WS_FTP.ini
/WinSCP.ini
/Wq_StranJF.js
/a2billing/admin/Public/PP_error.php
/a2billing/admin/Public/PP_error.php?c=accessdenied
/a2billing/customer/templates/default/footer.tpl
/about.jsp
/acenter/bottom.action
/acenter/images/banner_l.jpg
/acenter/index.action
/action-login
/action-site-type-link.html
/action-site-type-map
/addons/theme/stv1/_static/image/favicon.ico
/addons/theme/stv1/_static/ts2/layout.css
/addons/theme/stv2/_static/ts2/layout.css
/admin.php
/admin/SouthidcEditor/ButtonImage/standard/componentmenu.gif
/admin/SouthidcEditor/Dialog/dialog.js
/admin/SouthidcEditor/ewebeditor.asp
/admin/admin_login.php
/admin/assets/js/views/login.js
/admin/db/
/admin/images/logo.gif
/admin/images/logo_back.gif
/admin/inc/xml.xslt
/admin/index.php
/admin/js/IdSUtil.js
/admin/login.asp
/admin/login.aspx
/admin/login.php
/admin/pMA/
/admin/phpMyAdmin/
/admin/phpmyadmin/
/admin/sqladmin/
/admin/start/index.php
/admin/styles/default/main.css
/admin/sys/login.do
/admin/sysadmin/
/admin/template/article_more/config.htm
/admin/theme/web7/images/logo.png
/admin/web/
/admin_login.asp
/administrator
/administrator/PMA/
/administrator/images/logo.jpg
/administrator/manifests/files/joomla.xml
/administrator/phpMyAdmin/
/administrator/phpmyadmin/
/administrator/pma/
/adminsoft/templates/images/login_bg_top.jpg
/adminzone
/advfile/ad12.js
/api/api_user.xml
/app/Tpl/fanwe_1/js/DD_belatedPNG_0.0.8a-min.js
/app/home/skins/default/style.css
/app/images/login/logo.png
/app/images/login/toplogo.gif
/app/js/source/wcmlib/WCMConstants.js
/app/login.jsp
/apps/admin/_static/image/login_box_bg.png
/archive/archive.css
/archiver
/archiver/
/asp.net/README.txt
/assets/components/gallery/connector.php
/auth/login
/axis2/
/back/scripts/jspxcms_choose.js
/backup/bitcoin/
/backup/bitcoin/wallet.dat
/base/login/login.php
/batch.search.php
/bencandy.php
/bitcoin/
/bitcoin/backup/wallet.dat
/bitcoin/wallet.dat
/blog//wp-login.php
/blog/wp-index.php
/boards/
/bower.json
/cctrl/admin/ad_login.php
/cctrl/admin/images/logo.jpg
/cgi/index.cgi
/changelog.txt
/ckeditor/ckeditor.js
/ckeditor/ckfinder/ckfinder.html
/ckeditor/ckfinder/install.txt
/ckfinder/ckfinder.html
/ckfinder/install.txt
/client/en/community/supportcli.php
/clientscript/vbulletin_ajax_htmlloader.js
/cms/
/cms/lang/ru_utf8/css/sbIndex.css
/cms/leadermail/p_replydetail.jsp
/cms/webback/img/pic_login.jpg
/common/common.js
/common/help/images/helplogo.gif
/common/help/images/helplogo_zh.gif
/common/images/main/login/TRS-WCM.gif
/community/
/community/forum/
/connectors/system/phpthumb.php
/console/auth/reg_newuser.jsp
/console/include/not_login.htm
/console/js/CTRSRequestParam.js
/console/js/CWCMDialogHead.js
/core/docs/changelog.txt
/coremail/common/help/images/helplogo.gif
/coremail/common/help/images/helplogo_zh.gif
/coremail/displayVerifyCode.jsp
/coremail/forgetpwd.jsp
/css/graphics/icons/SAP_logo.gif
/current_config/passwd
/currentsetting.htm
/customdir/images/english_logo.jpg
/data/admin/ver.txt
/data/admin/verifies.txt
/data/config.js
/data/install.lock
/db/db-admin/
/db/dbadmin/
/db/myadmin/
/db/phpMyAdmin-3/
/db/phpMyAdmin/
/db/phpMyAdmin3/
/db/phpmyadmin/
/db/phpmyadmin3/
/db/webadmin/
/db/websql/
/dbadmin/
/default/css/em_css.css
/default/images/logo.gif
/deployment-config.json
/deptWebsiteAction.do
/design/header/oid_header.php
/dialog/dialog.js
/digg.php
/discussion/
/docs.css
/docs/
/docs/DOCUMENTATION.txt
/doku.php
/downloader/
/e/master/login.aspx
/e/search/index.php
/ecdomain/login.do
/ecdomain/portal/portlets/poll/js/poll.js
/editor/fckeditor.js
/editor/js/fckeditorcode_ie.js
/esbclient/login.php
/etm/indy.css
/evox/about
/examples/file-manager.html
/examples/index.html
/extern.php
/extman/default/images/logo.gif
/fck/editor/dialog/vote_chose.html
/fckeditor.js
/fckeditor/editor/dtd/fck_dtd_test.html
/fckeditor/editor/js/fckeditorcode_ie.js
/fckeditor/fckconfig.js
/fckeditor/fckeditor.js
/fckeditor/license.txt
/fcktemplates.xml
/feed.asp
/filezilla.xml
/forum.php
/forum/
/forum/forum.php
/forums/
/forums/forum.php
/forums/list.page
/fsmcms/cms/leadermail/p_replydetail.jsp
/ftpsync.settings
/heeroa/image/skin/0/copy_logo.gif
/help/ch_gb/images/help-title.gif
/help/user/index.html
/helpnew/faq/faq_simple_zh_CN.jsp
/hep/images/index/logo.gif
/hep/user/login.jsp
/history.txt
/htaccess.txt
/id_dsa
/id_rsa
/ids/admin/login.jsp
/ids/admin/userhome/forgetPwd.jsp
/image/zzcms-color.gif
/images/2_11.gif
/images/ASK_logo.gif
/images/App/Simple/bj.JPG
/images/actcms.css
/images/branding/logo.gif
/images/default/post_bt.gif
/images/dl_r1_c1.jpg
/images/favicon.ico
/images/fe_logo.png
/images/hwem.css
/images/index/5001/eoffice.gif
/images/js/common.js
/images/login/eyoumail.gif
/images/login/icon-up.gif
/images/login/logo.gif
/images/login9/login_33.jpg
/images/logo_88x31.gif
/images/tongda.ico
/images/zh-CN/logo.ico
/imagesschool/style1/flash2.jpg
/img/logo-zh_CN.swf
/img/pic/login/top-left.jpg
/inc/Templates/rss.xslt
/inc/common.js
/inc/install/License.html
/inc/playerKinds.xml
/inc/rsd.php
/inc/upload/upload.js
/include/dialog/config.php
/includes/general.js
/index.cgi
/index.php
/index.php?m=admin
/index.php?m=admin&c=index&a=login&pc_hash=
/index.php?m=link
/index.php?m=search
/index.php?m=wap
/install
/install/index.asp
/install/index.php
/issmall/
/jcms/index.jsp
/jcms/index_jcms.jsp
/jis/front/themes/blue/tpl/images/sso_01.gif
/jis/login.jsp
/jphoto/images/login/bg.jpg
/jphoto/images/login/login_loginbg.gif
/jphoto/index.jsp
/jphoto/jphoto/front/foot.jsp
/js/ajax_x.js
/js/buttons.js
/js/config.js
/js/jscolor/jscolor.js
/js/turboui.js
/jscripts/bbcodes_sceditor.js
/jscripts/select2/select2.css
/jsearch/
/jsearch/images/jsearch_logo.gif
/jsearch/search-index.jsp
/jsearch/template/1/images/jsearch_logo.gif
/jvideo/index.html
/jvideo/setup/images/jcmsmain_01.jpg
/jvideo/video/home/index.jsp
/kindeditor-min.js
/kindeditor.js
/kingdee/login/images/ctop_logo.gif
/ks_inc/ajax.js
/lang/en.js
/lfm.php
/lib/js/sdcms.book.js
/libs/js/iframe.js
/list.php
/login.asp
/login.aspx
/login/Login.jsp
/login/login.php
/m
/main.jsp
/maintlogin.jsp
/manager/html
/manager/templates/default/welcome.tpl
/master/login.aspx
/member/space/company/info.txt
/member/template/images/login.css
/messageboard/
/modules/Users/login.js
/mscms/css/main.css
/mthemes/default/images/logo.gif
/myadmin/
/mysql-admin/
/mysql/admin/
/mysql/dbadmin/
/mysql/mysqlmanager/
/mysql/pMA/
/mysql/pma/
/mysql/sqlmanager/
/mysqladmin/
/mysqlmanager/
/nc/images/UFTITLE.gif
/netcat/
/new_gb/help/images/usage/3.3.gif
/news/admin/Images/login.png
/news/admin/login.aspx
/next/img/logo.gif
/nmaplowercheck1536872109
/nmaplowercheck1536891816
/nz0808/index.asp
/oa/help/login.jpg
/oa/image/skin/0/copy_logo.gif
/oa/themes/mskin/login/login.jsp
/otsmobile/app/mgs/mgw.htm
/php-my-admin/
/php-myadmin/
/phpMyAdmin-3/
/phpMyAdmin/
/phpMyAdmin2/
/phpMyAdmin3/
/phpMyAdmin4/
/phpMyadmin/
/phpmanager/
/phpmy-admin/
/phpmy/
/phpmyAdmin/
/phpmyadmin
/phpmyadmin/
/phpmyadmin/docs.css
/phpmyadmin/favicon.ico
/phpmyadmin/themes/original/img/logo_right.png
/phpmyadmin2/
/phpmyadmin2011/
/phpmyadmin2012/
/phpmyadmin2013/
/phpmyadmin2014/
/phpmyadmin2015/
/phpmyadmin2016/
/phpmyadmin2017/
/phpmyadmin2018/
/phpmyadmin3/
/phpmyadmin4/
/phppma/
/piw/Images/log2.jpg
/piw/Login.jsp
/plug/publish
/plugin.php
/plugins/anchor/anchor.js
/plugins/filemanager/filemanager/js
/plus/download.php
/pma
/pma/
/pma2011/
/pma2012/
/pma2013/
/pma2014/
/pma2015/
/pma2016/
/pma2017/
/pma2018/
/pub/guiedit/guiedit.js
/pub/skins/pmwiki/pmwiki.css
/public/js/ipb.js
/rss.aspx
/rss.php
/scan?id=34&token=4343d
/script/login.js
/script/valid_formdata.js
/sdk
/seacms/data/admin/ver.txt
/search.html
/server/page_download/
/sftp-config.json
/sitemanager.xml
/siteserver/login.aspx
/siteserver/upgrade/default.aspx
/skin/admin/core/cluster/logo.gif
/skin/frontend/default/modern/css/styles.css
/sp/login
/sql/myadmin/
/sql/php-myadmin/
/sql/phpMyAdmin/
/sql/phpMyAdmin2/
/sql/phpmanager/
/sql/phpmy-admin/
/sql/phpmyadmin2/
/sql/sql-admin/
/sql/sqladmin/
/sql/sqlweb/
/sql/webadmin/
/sql/webdb/
/sql/websql/
/sqlmanager/
/sqoa/image/skin/0/copy_logo.gif
/startPage
/statics/admin/js/content_addtop.js
/structure/index.htm
/style/default/hdwiki.css
/stylesheet.css
/system/Login.aspx
/system/language/zh-cn.xml
/template/1/bluewise/_files/jspxcms.css
/template/2010/css/share.css
/template/default/index.html
/template/home.htm
/templates/default/css/cmstop-common.css
/templates/jsn_glass_pro/ext/hikashop/jsn_ext_hikashop.css
/test_404_page/
/theme/default/js/sdcms.js
/themes/default/default.css
/themes/default/graphics/favicon.ico
/themes/default/graphics/horde-power1.png
/themes/graphics/horde-power1.png
/tools/rss.aspx
/tpl/images/cmsloginui.png
/tpl/login/user/images/login_bg_1.jpg
/tpl/user/tpl1/css/skins/blue.css
/try
/u8qx/Tindex_cj/logoIndex.png
/uapws/resource/images/logo.png
/user/password
/user/password?name[%23post_render][0]=printf&name[%23markup]=ABCZ%0A
/user/register?element_parents=account/mail/%23value&ajax_form=1&_wrapper_format=drupal_ajax
/users
/util/login.aspx
/v3.0/web/assets/css/site.css
/vb/
/vbulletin/
/vc/images/alert.gif
/vc/main/images/t_logo.jpg
/vc/main/images/t_logo.jpg/UserCenter/css/admin/bgimg/admin_all_bg.png
/vc/vc/index_menu.jsp
/vc/vc/para/que_para.jsp
/vcms/admin.do
/vcms/favicon.ico
/vcms/templates/images/newcms/login_logo.jpg
/view/admin/menu.html
/view/admin/stat.html
/vipchat/home/site/1/images/vipchat_03.gif
/vipchat/home/site/1/images/vipchat_blue_01.jpg
/vipchat/setup/images/bg_q_x.gif
/vipchat/setup/index.htm
/wallet.dat
/wallet/
/wallet/wallet.dat
/was5/web/index.jsp
/wcm.files/js/browser.js
/web/theme/default2/reg.jsp
/web2/login_template/1.files/Logo1.jpg
/webbuilder/script/locale/wb-lang-zh_CN.js
/webout/theme/default2/reg.jsp
/whir_system/login.aspx
/whir_system/module/security/login.aspx
/winscp.ini
/wls-wsat/CoordinatorPortType
/wordpress//wp-login.php
/wordpress/wp-includes/
/wp-admin/
/wp-includes/
/wp-login.php
/wp//wp-login.php
/wp/wp-includes/
/wps/PA_PABJCSGENERALPROJE/js/location.js
/ws_ftp.ini
/xmlrpc.php
/ycportal/js/wbTextBox/showimg.jsp
/ymail/images/index_r1_c4.jpg
/yyoa/Navigation_help/A6_fuz/style/images/index_eye.gif
index.php?-s

Relevant Technology

Django Middleware on any platform

Complexity and required time

Complexity

One could argue that an Advanced level of complexity is helpful to prevent Webpage slow downs as there are components in the Django Middleware that may be seen by every web request.

Required time (ETA)

ShubhamTatvamasi commented 5 years ago

Can someone tell me how to block all these paths using nginx?