search

Arachni / arachni

Web Application Security Scanner Framework
http://www.arachni-scanner.com
Other
3.75k stars 760 forks source link

Error while scanning: [ArgumentError] Missing :url #488

Closed pirxthepilot closed 9 years ago

pirxthepilot commented 9 years ago

Hello, I am trying out Arachni in my lab environment. While scanning a test website, I got this message:

"This scan has the logged the following errors (you may want to report them):"

The scan is still ongoing as I write this.

Complete log below. Thanks!

pirxthepilot commented 9 years ago 
2014-10-14 18:37:32 +0800 --------------------------------------------------------------------------------
ENV:
---
HOSTNAME: openvaslab.wester.os
SELINUX_ROLE_REQUESTED: ''
GEM_HOME: "/home/joon/arachni-1.0.3-0.5.4/system/gems"
TERM: screen
SHELL: "/bin/bash"
HISTSIZE: '1000'
IRBRC: "/home/joon/arachni-1.0.3-0.5.4/bin/../system/usr/lib/ruby/.irbrc"
SSH_CLIENT: 192.168.186.1 60138 22
SELINUX_USE_CURRENT_RANGE: ''
MY_RUBY_HOME: "/home/joon/arachni-1.0.3-0.5.4/bin/../system/usr/lib/ruby"
SSH_TTY: "/dev/pts/0"
USER: joon
LD_LIBRARY_PATH: "/home/joon/arachni-1.0.3-0.5.4/bin/../system/usr/lib"
LS_COLORS: 'rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;05;37;41:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lz=01;31:*.xz=01;31:*.bz2=01;31:*.tbz=01;31:*.tbz2=01;31:*.bz=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.rar=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.jpg=01;35:*.jpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.axv=01;35:*.anx=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.axa=01;36:*.oga=01;36:*.spx=01;36:*.xspf=01;36:'
TERMCAP: "SC|screen|VT 100/ANSI X3.64 virtual terminal:\\\n\t:DO=\\E[%dB:LE=\\E[%dD:RI=\\E[%dC:UP=\\E[%dA:bs:bt=\\E[Z:\\\n\t:cd=\\E[J:ce=\\E[K:cl=\\E[H\\E[J:cm=\\E[%i%d;%dH:ct=\\E[3g:\\\n\t:do=^J:nd=\\E[C:pt:rc=\\E8:rs=\\Ec:sc=\\E7:st=\\EH:up=\\EM:\\\n\t:le=^H:bl=^G:cr=^M:it#8:ho=\\E[H:nw=\\EE:ta=^I:is=\\E)0:\\\n\t:li#44:co#168:am:xn:xv:LP:sr=\\EM:al=\\E[L:AL=\\E[%dL:\\\n\t:cs=\\E[%i%d;%dr:dl=\\E[M:DL=\\E[%dM:dc=\\E[P:DC=\\E[%dP:\\\n\t:im=\\E[4h:ei=\\E[4l:mi:IC=\\E[%d@:ks=\\E[?1h\\E=:\\\n\t:ke=\\E[?1l\\E>:vi=\\E[?25l:ve=\\E[34h\\E[?25h:vs=\\E[34l:\\\n\t:ti=\\E[?1049h:te=\\E[?1049l:us=\\E[4m:ue=\\E[24m:so=\\E[3m:\\\n\t:se=\\E[23m:mb=\\E[5m:md=\\E[1m:mr=\\E[7m:me=\\E[m:ms:\\\n\t:Co#8:pa#64:AF=\\E[3%dm:AB=\\E[4%dm:op=\\E[39;49m:AX:\\\n\t:vb=\\Eg:G0:as=\\E(0:ae=\\E(B:\\\n\t:ac=\\140\\140aaffggjjkkllmmnnooppqqrrssttuuvvwwxxyyzz{{||}}~~..--++,,hhII00:\\\n\t:po=\\E[5i:pf=\\E[4i:k0=\\E[10~:k1=\\EOP:k2=\\EOQ:k3=\\EOR:\\\n\t:k4=\\EOS:k5=\\E[15~:k6=\\E[17~:k7=\\E[18~:k8=\\E[19~:\\\n\t:k9=\\E[20~:k;=\\E[21~:F1=\\E[23~:F2=\\E[24~:F3=\\E[1;2P:\\\n\t:F4=\\E[1;2Q:F5=\\E[1;2R:F6=\\E[1;2S:F7=\\E[15;2~:\\\n\t:F8=\\E[17;2~:F9=\\E[18;2~:FA=\\E[19;2~:kb=\x7F:K2=\\EOE:\\\n\t:kB=\\E[Z:kF=\\E[1;2B:kR=\\E[1;2A:*4=\\E[3;2~:*7=\\E[1;2F:\\\n\t:#2=\\E[1;2H:#3=\\E[2;2~:#4=\\E[1;2D:%c=\\E[6;2~:%e=\\E[5;2~:\\\n\t:%i=\\E[1;2C:kh=\\E[1~:@1=\\E[1~:kH=\\E[4~:@7=\\E[4~:\\\n\t:kN=\\E[6~:kP=\\E[5~:kI=\\E[2~:kD=\\E[3~:ku=\\EOA:kd=\\EOB:\\\n\t:kr=\\EOC:kl=\\EOD:km:"
MAIL: "/var/spool/mail/joon"
PATH: "/home/joon/arachni-1.0.3-0.5.4/system/gems/bin:/home/joon/arachni-1.0.3-0.5.4/bin/../system/../bin:/home/joon/arachni-1.0.3-0.5.4/bin/../system/usr/bin:/home/joon/arachni-1.0.3-0.5.4/bin/../system/gems/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/joon/bin"
STY: 1590.pts-0.openvaslab
PWD: "/home/joon/arachni-1.0.3-0.5.4"
ARACHNI_WEBUI_LOGDIR: "/home/joon/arachni-1.0.3-0.5.4/bin/../system/logs/webui"
LANG: en_US.UTF-8
ARACHNI_FRAMEWORK_LOGDIR: "/home/joon/arachni-1.0.3-0.5.4/bin/../system/logs/framework"
SELINUX_LEVEL_REQUESTED: ''
HISTCONTROL: ignoredups
SHLVL: '2'
HOME: "/home/joon"
RAILS_ENV: production
DYLD_LIBRARY_PATH: "/home/joon/arachni-1.0.3-0.5.4/bin/../system/usr/lib:"
LOGNAME: joon
WINDOW: '1'
GEM_PATH: "/home/joon/arachni-1.0.3-0.5.4/bin/../system/gems"
SSH_CONNECTION: 192.168.112.1 60138 192.168.112.133 22
LESSOPEN: "|/usr/bin/lesspipe.sh %s"
RUBYLIB: "/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/bundler-1.7.3/lib:/home/joon/arachni-1.0.3-0.5.4/bin/../system/usr/lib/ruby:/home/joon/arachni-1.0.3-0.5.4/bin/../system/usr/lib/ruby/site_ruby/2.1.0:/home/joon/arachni-1.0.3-0.5.4/bin/../system/usr/lib/ruby/2.1.0:/home/joon/arachni-1.0.3-0.5.4/bin/../system/usr/lib/ruby/2.1.0/x86_64-linux:/home/joon/arachni-1.0.3-0.5.4/bin/../system/usr/lib/ruby/site_ruby/2.1.0/x86_64-linux"
RUBY_VERSION: ruby-2.1.1
G_BROKEN_FILENAMES: '1'
RACK_ENV: development
BUNDLE_GEMFILE: "/home/joon/arachni-1.0.3-0.5.4/system/arachni-ui-web/Gemfile"
_ORIGINAL_GEM_PATH: "/home/joon/arachni-1.0.3-0.5.4/bin/../system/gems"
BUNDLE_BIN_PATH: "/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/bundler-1.7.3/bin/bundle"
RUBYOPT: "-rbundler/setup"
MANPATH: "/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/kramdown-1.4.1/man"
BUNDLE_ORIG_MANPATH: "/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/kramdown-1.4.1/man"
--------------------------------------------------------------------------------
OPTIONS:
---
scope:
  redundant_path_patterns: {}
  dom_depth_limit: 10
  exclude_path_patterns: []
  exclude_content_patterns: []
  include_path_patterns: []
  restrict_paths: []
  extend_paths: []
  url_rewrites: {}
session: {}
input:
  values:
    "(?i-mx:name)": arachni_name
    "(?i-mx:user)": arachni_user
    "(?i-mx:usr)": arachni_user
    "(?i-mx:pass)": 5543!%arachni_secret
    "(?i-mx:txt)": arachni_text
    "(?i-mx:num)": '132'
    "(?i-mx:amount)": '100'
    "(?i-mx:mail)": arachni@email.gr
    "(?i-mx:account)": '12'
    "(?i-mx:id)": '1'
  default_values:
    "(?i-mx:name)": arachni_name
    "(?i-mx:user)": arachni_user
    "(?i-mx:usr)": arachni_user
    "(?i-mx:pass)": 5543!%arachni_secret
    "(?i-mx:txt)": arachni_text
    "(?i-mx:num)": '132'
    "(?i-mx:amount)": '100'
    "(?i-mx:mail)": arachni@email.gr
    "(?i-mx:account)": '12'
    "(?i-mx:id)": '1'
  without_defaults: true
  force: false
audit:
  exclude_vector_patterns: []
  include_vector_patterns: []
  link_templates: []
  links: true
  forms: true
  cookies: true
datastore:
  token: 70f47a2ec4085a4018744d2327df9973
http:
  user_agent: Arachni/v1.0.3
  request_timeout: 50000
  request_redirect_limit: 5
  request_concurrency: 20
  request_queue_size: 500
  request_headers: {}
  cookies: {}
browser_cluster:
  pool_size: 6
  job_timeout: 120
  worker_time_to_live: 100
  ignore_images: false
  screen_width: 1600
  screen_height: 1200
checks:
- allowed_methods
- backdoors
- backup_directories
- backup_files
- captcha
- code_injection
- code_injection_php_input_wrapper
- code_injection_timing
- common_directories
- common_files
- cookie_set_for_parent_domain
- credit_card
- csrf
- cvs_svn_users
- directory_listing
- emails
- file_inclusion
- form_upload
- hsts
- htaccess_limit
- html_objects
- http_only_cookies
- http_put
- insecure_cookies
- interesting_responses
- ldap_injection
- localstart_asp
- mixed_resource
- no_sql_injection
- no_sql_injection_differential
- origin_spoof_access_restriction_bypass
- os_cmd_injection
- os_cmd_injection_timing
- password_autocomplete
- path_traversal
- private_ip
- response_splitting
- rfi
- session_fixation
- source_code_disclosure
- sql_injection
- sql_injection_differential
- sql_injection_timing
- ssn
- trainer
- unencrypted_password_forms
- unvalidated_redirect
- webdav
- xpath_injection
- xss
- xss_dom
- xss_dom_inputs
- xss_dom_script_context
- xss_event
- xss_path
- xss_script_context
- xss_tag
- xst
platforms: []
plugins:
  autothrottle: {}
  discovery: {}
  healthmap: {}
  timing_attacks: {}
  uniformity: {}
no_fingerprinting: false
authorized_by: 
url: http://dvwa/
--------------------------------------------------------------------------------
[2014-10-14 18:37:32 +0800] Client: [ArgumentError] Missing :url.
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/http/message.rb:49:in `initialize'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/http/response.rb:65:in `initialize'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/http/response.rb:207:in `new'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/http/response.rb:207:in `from_typhoeus'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/http/request.rb:370:in `block in to_typhoeus'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/typhoeus-0.6.9/lib/typhoeus/request/callbacks.rb:129:in `call'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/typhoeus-0.6.9/lib/typhoeus/request/callbacks.rb:129:in `block in execute_callbacks'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/typhoeus-0.6.9/lib/typhoeus/request/callbacks.rb:128:in `map'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/typhoeus-0.6.9/lib/typhoeus/request/callbacks.rb:128:in `execute_callbacks'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/typhoeus-0.6.9/lib/typhoeus/request/operations.rb:35:in `finish'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/typhoeus-0.6.9/lib/typhoeus/easy_factory.rb:105:in `block in set_callback'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/ethon-0.7.1/lib/ethon/easy/response_callbacks.rb:65:in `call'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/ethon-0.7.1/lib/ethon/easy/response_callbacks.rb:65:in `block in complete'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/ethon-0.7.1/lib/ethon/easy/response_callbacks.rb:65:in `each'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/ethon-0.7.1/lib/ethon/easy/response_callbacks.rb:65:in `complete'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/ethon-0.7.1/lib/ethon/multi/operations.rb:148:in `check'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/ethon-0.7.1/lib/ethon/multi/operations.rb:161:in `run'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/ethon-0.7.1/lib/ethon/multi/operations.rb:43:in `perform'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/typhoeus-0.6.9/lib/typhoeus/hydra/runnable.rb:15:in `run'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/typhoeus-0.6.9/lib/typhoeus/hydra/memoizable.rb:51:in `run'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/http/client.rb:695:in `hydra_run'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/http/client.rb:206:in `block in run'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/utilities.rb:395:in `call'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/utilities.rb:395:in `exception_jail'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/http/client.rb:202:in `run'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/framework.rb:1186:in `harvest_http_responses'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/framework.rb:312:in `audit_page'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/framework.rb:1109:in `audit_queues'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/rpc/server/framework/multi_instance.rb:221:in `audit_queues'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/framework.rb:1048:in `block in audit'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/framework.rb:1028:in `loop'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/framework.rb:1028:in `audit'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/framework.rb:224:in `block in run'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/utilities.rb:395:in `call'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/utilities.rb:395:in `exception_jail'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/framework.rb:224:in `run'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/rpc/server/framework.rb:154:in `block in run'
[2014-10-14 18:37:32 +0800] Client: 
[2014-10-14 18:37:32 +0800] Client: Parent:
[2014-10-14 18:37:32 +0800] Client: Arachni::HTTP::Client
[2014-10-14 18:37:32 +0800] Client: 
[2014-10-14 18:37:32 +0800] Client: Block:
[2014-10-14 18:37:32 +0800] Client: #<Proc:0x007f3a8f30c868@/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/http/client.rb:202>
[2014-10-14 18:37:32 +0800] Client: 
[2014-10-14 18:37:32 +0800] Client: Caller:
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/utilities.rb:395:in `exception_jail'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/http/client.rb:202:in `run'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/framework.rb:1186:in `harvest_http_responses'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/framework.rb:312:in `audit_page'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/framework.rb:1109:in `audit_queues'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/rpc/server/framework/multi_instance.rb:221:in `audit_queues'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/framework.rb:1048:in `block in audit'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/framework.rb:1028:in `loop'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/framework.rb:1028:in `audit'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/framework.rb:224:in `block in run'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/utilities.rb:395:in `call'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/utilities.rb:395:in `exception_jail'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/framework.rb:224:in `run'
[2014-10-14 18:37:32 +0800] Client: /home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/rpc/server/framework.rb:154:in `block in run'
[2014-10-14 18:37:32 +0800] Client: --------------------------------------------------------------------------------
Zapotek commented 9 years ago

Hello there,

Looks like an HTTP response has a URL that Arachni can't handle, which is strange since the URL would have first been sanitized by Typhoeus/Ethon/libCURL.

This only happens when a website is vulnerable to an unvalidated redirect issue, as some of the audit payloads may force the webapp to return a redirect with an invalid URL. This error doesn't cause any problems, but it happens at the guts of the system and so I haven't found a nice way to handle it.

Still, just to be sure, I'll run it against DVWA to see exactly what's going on.

Cheers

PS. Might take me a few days, I'm a bit under the weather,

pirxthepilot commented 9 years ago

Hi Tasos, thanks for the quick response! Take your time, I am content knowing that the error can be ignored.

So finally the scan stopped unexpectedly and I got additional errors (which may or may not have caused the scan to stop?)

Too many open files @ rb_sysopen - /home/joon/arachni-1.0.3-0.5.4/bin/../system/logs/framework//Instance - 1825-39985.error.log
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/rpc/server/framework/multi_instance.rb:47:in `read'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/rpc/server/framework/multi_instance.rb:47:in `errors'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/rpc/server/framework/multi_instance.rb:121:in `progress'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/rpc/server/instance.rb:726:in `progress_handler'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/rpc/server/instance.rb:438:in `native_progress'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-rpc-0.2.1.1/lib/arachni/rpc/server.rb:207:in `call'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-rpc-0.2.1.1/lib/arachni/rpc/server/handler.rb:56:in `receive_request'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-rpc-0.2.1.1/lib/arachni/rpc/server/handler.rb:95:in `receive_object'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-rpc-0.2.1.1/lib/arachni/rpc/protocol.rb:52:in `on_read'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor/connection.rb:236:in `block in _read'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor/connection/error.rb:26:in `call'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor/connection/error.rb:26:in `translate'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor/connection.rb:235:in `_read'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor.rb:568:in `each'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor.rb:568:in `block in process_connections'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor.rb:568:in `each'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor.rb:568:in `process_connections'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor.rb:317:in `block in run'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor.rb:308:in `loop'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor.rb:308:in `run'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/rpc/server/instance.rb:152:in `initialize'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/processes/executables/instance.rb:13:in `new'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/processes/executables/instance.rb:13:in `<top (required)>'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/processes/executables/base.rb:9:in `load'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-1.0.3/lib/arachni/processes/executables/base.rb:9:in `<main>'
--------------------------------------------------------------------------------
/home/joon/arachni-1.0.3-0.5.4/system/arachni-ui-web/app/models/scan.rb:523:in `block in refresh'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-rpc-0.2.1.1/lib/arachni/rpc/proxy.rb:58:in `call'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-rpc-0.2.1.1/lib/arachni/rpc/proxy.rb:58:in `block (2 levels) in translate'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-rpc-0.2.1.1/lib/arachni/rpc/client/handler.rb:77:in `call'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-rpc-0.2.1.1/lib/arachni/rpc/client/handler.rb:77:in `receive_response'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-rpc-0.2.1.1/lib/arachni/rpc/client/handler.rb:137:in `receive_object'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-rpc-0.2.1.1/lib/arachni/rpc/protocol.rb:52:in `on_read'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor/connection.rb:236:in `block in _read'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor/connection/error.rb:26:in `call'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor/connection/error.rb:26:in `translate'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor/connection.rb:235:in `_read'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor.rb:568:in `each'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor.rb:568:in `block in process_connections'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor.rb:568:in `each'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor.rb:568:in `process_connections'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor.rb:317:in `block in run'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor.rb:308:in `loop'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor.rb:308:in `run'
/home/joon/arachni-1.0.3-0.5.4/system/gems/gems/arachni-reactor-0.1.0.beta5/lib/arachni/reactor.rb:350:in `block in run_in_thread'
********************************************************************************
Zapotek commented 9 years ago

Yeah that is a crash, but it's got more to do with your OS resource limits rather than a bug in Arachni (most probably), can you please show me the output of ulimit -a?

pirxthepilot commented 9 years ago

Here you go:

[root@lab joon]# ulimit -a
core file size          (blocks, -c) 0
data seg size           (kbytes, -d) unlimited
scheduling priority             (-e) 0
file size               (blocks, -f) unlimited
pending signals                 (-i) 7763
max locked memory       (kbytes, -l) 64
max memory size         (kbytes, -m) unlimited
open files                      (-n) 1024
pipe size            (512 bytes, -p) 8
POSIX message queues     (bytes, -q) 819200
real-time priority              (-r) 0
stack size              (kbytes, -s) 10240
cpu time               (seconds, -t) unlimited
max user processes              (-u) 7763
virtual memory          (kbytes, -v) unlimited
file locks                      (-x) unlimited
Zapotek commented 9 years ago

The open files limit isn't unusually low, on the other hand, it's not like there's a clear expectation on how many files Arachni will need to have open at any given time (especially since network connections count as open files as well).

For now, can you please increase your OS's limit to 4 times its current value? When I feel better I'll run some tests for that too, make sure there aren't any file descriptor leaks.

pirxthepilot commented 9 years ago

Hi Tasos, I have set the limit to 65536 then ran a scan again on a different test site (dojo-basic this time). I left it overnight and was still running in the morning. I stopped the scan - total runtime was ~8 hours, no errors.

I then doubled the RAM on the VM where arachni is installed, to 2GB. In arachni, I changed concurrency to 10, timeout to 15 sec, and queue size to 300.

Then I ran another scan on one of our development websites and left it overnight again. This time it finished after 10 hours (also no errors). So that's solved then!

Was wondering if you can provide some tips to significantly reduce the scan time, but without disabling features that would lessen the value of the scan (e.g. all OWASP top 10 vulns should be included).

Thanks a lot!

Zapotek commented 9 years ago

There are things you can do to increase performance but it's going to be a trade-off with resource usage, see: http://support.arachni-scanner.com/kb/general-use/optimizing-for-faster-scans

Glad to hear the increased FDs solved the issue, although I'll still have a look at it to make sure nothing fishy is going on.