Feature request: IgnoreComments advanced option for word list processing

snoski3 commented 5 years ago

I'm not sure if this should be under feature request or a bug.

Steps to reproduce

How'd you do it?

Use aux/scanner/ssh/ssh_enumusers
set user_file to /usr/share/wordlists/dirbuster/apache-user-enum-1.0.txt

This section should also tell us any relevant information about the environment; for example, if an exploit that used to work is failing, tell us the victim operating system and service versions.

Expected behavior

What should happen? The scanner skips the comment section and continues to the list of words.

Current behavior

What happens instead? It uses the comment section as input.

System stuff

Metasploit version

Get this with the version command in msfconsole (or git log -1 --pretty=oneline for a source install). Framework: 5.0.11-dev Console : 5.0.11-dev

I installed Metasploit with:

[x ] Kali package via apt
[ ] Omnibus installer (nightly)
[ ] Commercial/Community installer (from http://www.rapid7.com/products/metasploit/download.jsp)
[ ] Source install (please specify ruby version)

OS

What OS are you running Metasploit on? Kali 2019.1 ssh enum

bcoles commented 5 years ago

This should probably be an advanced option IgnoreComments, off by default, so as to not unintentionally skip passwords beginning with a #.

An over-engineered approach may be to count the number of lines beginning with # and throw a warning. At which point it would already be too late, so, :shrug:

As far as I'm aware, no standard exists for word lists. Considering lines prefaced with # to be comments is an unofficial standard. For example, despite their frequent use in CSV documents, the CSV standard does not permit comments.

wvu commented 5 years ago

I saw your post to the mailing list, @snoski3. I agree it can be a useful enhancement, but @bcoles hit all of my own points right on the head.

At best, I'd make this a default-off advanced option. There is no standard for processing word lists, and I'm not keen to generalize niche behavior.

Also, note that this behavior isn't exclusive to ssh_enumusers. That may complicate implementation.

In the meantime, I suggest preprocessing the file traditionally. Here's what I did with grep(1):

wvu@kharak:~/Downloads$ grep -vE '^(#|$)' apache-user-enum-1.0.txt > apache-user-enum-1.0-preprocessed.txt
wvu@kharak:~/Downloads$ head !$
head apache-user-enum-1.0-preprocessed.txt
~root
~toor
~bin
~daemon
~adm
~lp
~sync
~shutdown
~halt
~mail
wvu@kharak:~/Downloads$

Note that blank lines are also removed. With a Unix pipeline, you can use your imagination.

github-actions[bot] commented 3 years ago

Hi!

This issue has been left open with no activity for a while now.

We get a lot of issues, so we currently close issues after 60 days of inactivity. It’s been at least 30 days since the last update here. If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request.

rapid7 / metasploit-framework