search

3xp10it / xcdn

Try to find out the real ip behind cdn
335 stars 114 forks source link

xcdn在get_http_or_https出错 #1

Closed d4wner closed 5 years ago

d4wner commented 6 years ago

老哥，不知道这项目还在维护没，看错误是解析出错了？ 使用命令应该没错， python3 xcdn.py www.baidu.com 环境3.5的，但是在解析url返回结果时报错了，不知道是exp10it库的问题还是啥原因。

Traceback (most recent call last):
  File "xcdn.py", line 286, in <module>
    Xcdn(domain)
  File "xcdn.py", line 58, in __init__
    self.http_or_https=get_http_or_https(self.domain)
  File "/usr/local/lib/python3.5/dist-packages/exp10it.py", line 3957, in get_http_or_https
    bing_record = get_http_or_https_from_search_engine(domain)
  File "/usr/local/lib/python3.5/dist-packages/exp10it.py", line 3928, in get_http_or_https_from_search_engine
    urls_list = collect_urls_from_url(url)['y1']
  File "/usr/local/lib/python3.5/dist-packages/exp10it.py", line 3796, in collect_urls_from_url
    title = has_title.group[1]
TypeError: 'builtin_function_or_method' object is not subscriptable
3xp10it commented 6 years ago

已更新exp10it，出错原因在这行： title = has_title.group[1] 应该是： title = has_title.group(1) 直接重新运行原来的命令即可。

d4wner commented 6 years ago

exp10it库运行时出新的幺蛾子了...git主劳烦看看：

现在从在线cloudflare类型cdn查询真实ip接口尝试获取真实ip
Traceback (most recent call last):
  File "xcdn.py", line 286, in <module>
    Xcdn(domain)
  File "xcdn.py", line 63, in __init__
    actual_ip = self.get_actual_ip_from_domain()
  File "xcdn.py", line 262, in get_actual_ip_from_domain
    ip_value = self.get_ip_value_from_online_cloudflare_interface()
  File "xcdn.py", line 240, in get_ip_value_from_online_cloudflare_interface
    content = post_request(url, post_data)
  File "/usr/local/lib/python3.5/dist-packages/exp10it.py", line 1576, in post_request
    if DELAY != "":
NameError: name 'DELAY' is not defined
3xp10it commented 5 years ago

已更新,现在应该没什么问题了

d4wner commented 5 years ago

额，再给git主发一次提问，直接运行还是报那个DELAY的错：

python3 xcdn.py www.baidu.com

另外换了个机器，重新clone，还pull了几次.，中途补装了个figlet，还是老样子。

3xp10it commented 5 years ago

这边运行python3 xcdn.py www.baidu.com是正常工作的，不知你的运行环境是什么样的？ 操作系统是不是kali?

d4wner commented 5 years ago

事实上我用了朋友的域名，他是cloudfare的cdn，www.xxxx.net博主试试，mac os和centos下试了都那个错，kali我这边环境有点问题。

3xp10it commented 5 years ago

我这边用kali是可以执行的，下面是执行记录：

⋊> ~/xcdn on master ⨯ py3 xcdn.py www.bstaint.net                                                
20:42:38
Requirement already up-to-date: exp10it in /usr/local/lib/python3.6/dist-packages
Requirement already up-to-date: bs4 in /usr/local/lib/python3.6/dist-packages (from exp10it)
Requirement already up-to-date: chardet in /usr/lib/python3/dist-packages (from exp10it)
Collecting colorama (from exp10it)
  Downloading https://files.pythonhosted.org/packages/0a/93/6e8289231675d561d476d656c2ee3a868c1cca207e16c118d4503b25e2bf/colorama-0.4.0-py2.py3-none-any.whl
  Requirement already up-to-date: configparser in /usr/local/lib/python3.6/dist-packages (from exp10it)
  Requirement already up-to-date: mechanicalsoup in /usr/local/lib/python3.6/dist-packages (from exp10it)
  Requirement already up-to-date: readline in /usr/local/lib/python3.6/dist-packages (from exp10it)
  Requirement already up-to-date: requests in /usr/local/lib/python3.6/dist-packages (from exp10it)
  Requirement already up-to-date: selenium in /usr/local/lib/python3.6/dist-packages (from exp10it)
  Requirement already up-to-date: wget in /usr/local/lib/python3.6/dist-packages (from exp10it)
  Requirement already up-to-date: beautifulsoup4 in /usr/local/lib/python3.6/dist-packages (from bs4->exp10it)
  Requirement already up-to-date: six>=1.4 in /usr/lib/python3/dist-packages (from mechanicalsoup->exp10it)
  Requirement already up-to-date: lxml in /usr/local/lib/python3.6/dist-packages (from mechanicalsoup->exp10it)
  Requirement already up-to-date: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->exp10it)
  Requirement already up-to-date: idna<2.8,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->exp10it)
  Requirement already up-to-date: urllib3<1.24,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->exp10it)
Installing collected packages: colorama
Found existing installation: colorama 0.3.9
Uninstalling colorama-0.3.9:
Successfully uninstalled colorama-0.3.9
Successfully installed colorama-0.4.0
 __ /        _ |   \_) |
 _ \\ \ / _ \ | (  ||  _|
 ___/ _\_\.__/_|\__/_|\__|
 _|
domain的http或https是:http
进入获取真实ip函数,认为每个domain都是有cdn的情况来处理
现在检测domain:www.bstaint.net是否有cdn
has_cdn=1 from ns,and cdn is cloudflare
检测到domain:www.bstaint.net的A记录不止一个,认为它有cdn
现在从在线cloudflare类型cdn查询真实ip接口尝试获取真实ip
恭喜,www.bstaint.net的真实ip是54.175.185.45
d4wner commented 5 years ago

刚kali我也试了下，还是前面那个错，也许是我缺啥东西吧，上了国外代理也不行。 我py3.5的，按理说应该没问题，也没有其他报错，绝了，这换了三个环境了，下面是我这边kali的结果。

Requirement already up-to-date: exp10it in /usr/local/lib/python3.5/dist-packages
Requirement already up-to-date: bs4 in /usr/local/lib/python3.5/dist-packages (from exp10it)
Requirement already up-to-date: chardet in /usr/local/lib/python3.5/dist-packages (from exp10it)
Requirement already up-to-date: colorama in /usr/local/lib/python3.5/dist-packages (from exp10it)
Requirement already up-to-date: configparser in /usr/local/lib/python3.5/dist-packages (from exp10it)
Requirement already up-to-date: mechanicalsoup in /usr/local/lib/python3.5/dist-packages (from exp10it)
Requirement already up-to-date: readline in /usr/local/lib/python3.5/dist-packages (from exp10it)
Requirement already up-to-date: requests in /usr/local/lib/python3.5/dist-packages (from exp10it)
Requirement already up-to-date: selenium in /usr/local/lib/python3.5/dist-packages (from exp10it)
Requirement already up-to-date: wget in /usr/local/lib/python3.5/dist-packages (from exp10it)
Requirement already up-to-date: beautifulsoup4 in /usr/local/lib/python3.5/dist-packages (from bs4->exp10it)
Requirement already up-to-date: six>=1.4 in /usr/local/lib/python3.5/dist-packages (from mechanicalsoup->exp10it)
Requirement already up-to-date: lxml in /usr/local/lib/python3.5/dist-packages (from mechanicalsoup->exp10it)
Requirement already up-to-date: certifi>=2017.4.17 in /usr/local/lib/python3.5/dist-packages (from requests->exp10it)
Requirement already up-to-date: urllib3<1.24,>=1.21.1 in /usr/local/lib/python3.5/dist-packages (from requests->exp10it)
Requirement already up-to-date: idna<2.8,>=2.5 in /usr/local/lib/python3.5/dist-packages (from requests->exp10it)
 #####                  #     ###           
#     # #    # #####   ##    #   #  # ##### 
      #  #  #  #    # # #   #     # #   #   
 #####    ##   #    #   #   #     # #   #   
      #   ##   #####    #   #     # #   #   
#     #  #  #  #        #    #   #  #   #   
 #####  #    # #      #####   ###   #   #   

domain的http或https是:http
进入获取真实ip函数,认为每个domain都是有cdn的情况来处理
现在检测domain:www.bstaint.net是否有cdn
has_cdn=1 from ns,and cdn is cloudflare
检测到domain:www.bstaint.net的A记录不止一个,认为它有cdn
现在从在线cloudflare类型cdn查询真实ip接口尝试获取真实ip
Traceback (most recent call last):
  File "xcdn.py", line 286, in <module>
    Xcdn(domain)
  File "xcdn.py", line 63, in __init__
    actual_ip = self.get_actual_ip_from_domain()
  File "xcdn.py", line 262, in get_actual_ip_from_domain
    ip_value = self.get_ip_value_from_online_cloudflare_interface()
  File "xcdn.py", line 240, in get_ip_value_from_online_cloudflare_interface
    content = post_request(url, post_data)
  File "/usr/local/lib/python3.5/dist-packages/exp10it.py", line 1576, in post_request
    if DELAY != "":
NameError: name 'DELAY' is not defined
3xp10it commented 5 years ago

应该是exp10it没有更新，kali下先运行下面命令更新下exp10it试试: pip3 install exp10it -U --no-cache-dir 然后运行: python3 xcdn.py www.bstaint.net

如果还不成功估计是你的exp10it模块的路径有多个，最先用到的exp10it不是更新后的文件。

d4wner commented 5 years ago

前两次是pull的最新的库版本，刚才按您的法子又更新了下，还是那样。 我locate了下kali的exp10it库路径，好像就一个版本v 2.6.95：

/usr/local/lib/python3.5/dist-packages/exp10it-2.6.95.egg-info
/usr/local/lib/python3.5/dist-packages/exp10it.py
/usr/local/lib/python3.5/dist-packages/__pycache__/exp10it.cpython-35.pyc
/usr/local/lib/python3.5/dist-packages/exp10it-2.6.95.egg-info/PKG-INFO
/usr/local/lib/python3.5/dist-packages/exp10it-2.6.95.egg-info/SOURCES.txt
/usr/local/lib/python3.5/dist-packages/exp10it-2.6.95.egg-info/dependency_links.txt
/usr/local/lib/python3.5/dist-packages/exp10it-2.6.95.egg-info/installed-files.txt
/usr/local/lib/python3.5/dist-packages/exp10it-2.6.95.egg-info/requires.txt
/usr/local/lib/python3.5/dist-packages/exp10it-2.6.95.egg-info/top_level.txt
/usr/local/lib/python3.5/dist-packages/exp10it-2.6.95.egg-info/zip-safe
3xp10it commented 5 years ago

试试运行这个命令： cd /tmp && git clone https://github.com/3xp10it/xcdn && cd /tmp/xcdn && wget https://raw.githubusercontent.com/3xp10it/exp10it/master/exp10it.py && python3 xcdn.py www.bstaint.net

d4wner commented 5 years ago

搞定了...不过博主能解释下再clone一次为啥就行了么.... 上周五我换了个两次环境，新clone下来，确实也报错，当时应该exp10it的库是最新的吧。

3xp10it commented 5 years ago

当时应该也是最新的，之前两次失败的具体原因估计不好追溯了。

y0rkl4u commented 5 years ago

我这也报 if DELAY != "": NameError: name 'DELAY' is not defined mac系统，重复git clone 也不行

3xp10it commented 5 years ago

@iliuyongkang 试试运行这个命令： cd /tmp && git clone https://github.com/3xp10it/xcdn && cd /tmp/xcdn && wget https://raw.githubusercontent.com/3xp10it/exp10it/master/exp10it.py && python3 xcdn.py www.bstaint.net