theharvester: No such file or directory: 'proxies.yaml'

kortac commented 1 year ago

`brew gist-logs <formula>` link OR `brew config` AND `brew doctor` output

➜  ~ brew config
HOMEBREW_VERSION: 3.6.11-5-g06b7573
ORIGIN: https://github.com/Homebrew/brew
HEAD: 06b7573e0b87eeb67907658d0004548b7c72c0d3
Last commit: 6 hours ago
Core tap ORIGIN: https://github.com/Homebrew/homebrew-core
Core tap HEAD: f5ea5df3d21bb60c81e05b1449e008d3db670d12
Core tap last commit: 5 minutes ago
Core tap branch: master
HOMEBREW_PREFIX: /opt/homebrew
HOMEBREW_CASK_OPTS: []
HOMEBREW_MAKE_JOBS: 8
Homebrew Ruby: 2.6.10 => /System/Library/Frameworks/Ruby.framework/Versions/2.6/usr/bin/ruby
CPU: octa-core 64-bit arm_firestorm_icestorm
Clang: 14.0.0 build 1400
Git: 2.37.1 => /Applications/Xcode.app/Contents/Developer/usr/bin/git
Curl: 7.84.0 => /usr/bin/curl
macOS: 13.0-arm64
CLT: 14.1.0.0.1.1666437224
Xcode: 14.1
Rosetta 2: false

➜  ~ brew doctor
Your system is ready to brew.

Verification

[X] I ran brew update and am still able to reproduce my issue.
[X] I have resolved all warnings from brew doctor and that did not fix my problem.

What were you trying to do (and why)?

Executing theharvester fails with an error message stating proxies.yaml cannot be found.

I tried the step-by-step reproduction on an iMac 2021 and a MacBook Pro 2021, both with Ventura. On both machines it fails.

What happened (include all command output)?

➜  ~ theharvester
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/theharvester/4.0.3_1/libexec/lib/python3.10/site-packages/theHarvester/lib/core.py", line 90, in proxy_list
    with open('/etc/theHarvester/proxies.yaml', 'r') as proxy_file:
FileNotFoundError: [Errno 2] No such file or directory: '/etc/theHarvester/proxies.yaml'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/theharvester/4.0.3_1/libexec/lib/python3.10/site-packages/theHarvester/lib/core.py", line 94, in proxy_list
    with open('/usr/local/etc/theHarvester/proxies.yaml', 'r') as proxy_file:
FileNotFoundError: [Errno 2] No such file or directory: '/usr/local/etc/theHarvester/proxies.yaml'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/homebrew/bin/theharvester", line 6, in <module>
    from theHarvester import __main__
  File "/opt/homebrew/Cellar/theharvester/4.0.3_1/libexec/lib/python3.10/site-packages/theHarvester/__main__.py", line 4, in <module>
    from theHarvester.discovery import dnssearch, takeover, shodansearch
  File "/opt/homebrew/Cellar/theharvester/4.0.3_1/libexec/lib/python3.10/site-packages/theHarvester/discovery/takeover.py", line 1, in <module>
    from theHarvester.lib.core import *
  File "/opt/homebrew/Cellar/theharvester/4.0.3_1/libexec/lib/python3.10/site-packages/theHarvester/lib/core.py", line 233, in <module>
    class AsyncFetcher:
  File "/opt/homebrew/Cellar/theharvester/4.0.3_1/libexec/lib/python3.10/site-packages/theHarvester/lib/core.py", line 234, in AsyncFetcher
    proxy_list = Core.proxy_list()
  File "/opt/homebrew/Cellar/theharvester/4.0.3_1/libexec/lib/python3.10/site-packages/theHarvester/lib/core.py", line 97, in proxy_list
    with open('proxies.yaml', 'r') as proxy_file:
FileNotFoundError: [Errno 2] No such file or directory: 'proxies.yaml'

What did you expect to happen?

Homebrew version of theharvester is looking for proxies.yaml in /etc/theHarvester instead of /opt/homebrew/etc/theharvester/.

Manually linking sudo ln -s /opt/homebrew/etc/theharvester /etc/theHarvester solves the issue:

➜  ~ sudo ln -s /opt/homebrew/etc/theharvester /etc/theHarvester
➜  ~ theharvester

*******************************************************************
*  _   _                                            _             *
* | |_| |__   ___    /\  /\__ _ _ ____   _____  ___| |_ ___ _ __  *
* | __|  _ \ / _ \  / /_/ / _` | '__\ \ / / _ \/ __| __/ _ \ '__| *
* | |_| | | |  __/ / __  / (_| | |   \ V /  __/\__ \ ||  __/ |    *
*  \__|_| |_|\___| \/ /_/ \__,_|_|    \_/ \___||___/\__\___|_|    *
*                                                                 *
* theHarvester 4.0.3                                              *
* Coded by Christian Martorella                                   *
* Edge-Security Research                                          *
* cmartorella@edge-security.com                                   *
*                                                                 *
******************************************************************* 

usage: theharvester [-h] -d DOMAIN [-l LIMIT] [-S START] [-g] [-p] [-s] [--screenshot SCREENSHOT] [-v] [-e DNS_SERVER] [-t DNS_TLD]
                    [-r] [-n] [-c] [-f FILENAME] [-b SOURCE]
theharvester: error: the following arguments are required: -d/--domain

I would expect brew theharvester to look for the yaml files in opt/homebrew/etc/theharvester instead of /etc/theHarvester.

Step-by-step reproduction instructions (by running `brew` commands)

brew install theharvester
theharvester

cho-m commented 1 year ago

Currently, various file paths are hardcoded, though the fallback path does work on Intel macOS (/usr/local/etc).

Best way would be for upstream to allow setting the particular path. Even something like an environment variable would help since Homebrew can create a wrapper to set the environment variable.

On Homebrew side, at best some type of workaround might be possible to manually find-and-replace either /etc/ or /usr/local/etc/ in some code. However, this can be fragile as we can't track if upstream adds new files, changes code, etc.

iMichka commented 1 year ago

Closing as this looks like an upstream issue.

Homebrew / homebrew-core