Feature suggestion

[Codeiology]

Can we make this compatible with other operating systems such as macOS, or Windows? I am a really big fan of this tool, and would love to see its availability globalized. I have tried running it on my macOS system, and, no matter how much I modify the code, I cannot get it to work. I figured I would reach out to the devs! Huge props, by the way.

[opsdisk]

Hi @Codeiology Can you provide some screenshots or log output to what you're seeing? As long as you have at least Python 3.6, it should be Operating System agnostic. There may be some odd os library or file path stuff I didn't account for though.

[Codeiology]

I also tried editing the code myself to try to fix it, but I just kept getting SSL problems and code 429. And when I tried to fix that by giving the program paths to cacert.pem, I just got even longer, much more daunting tracebacks.

[Codeiology]

Details about environment:

OS: macOS 13.3.1 22E261 arm64 CPU: Apple M1 Shell: zsh 5.9 Python version: Python 3.11.1 (Homebrew says I have 3.11.3) Package manager: Homebrew

All required pip3 modules installed with pip3 install -r requirements.txt not using virtual environment or externally managed environment

[opsdisk]

Thanks for that info @Codeiology

Couple more questions:

1) Are you cloning the repo and running the code? Or are you using the tool provided in the Kali OS?

2) Can you provide the full command you are running?

3) Any reason you don't want to use a virtual environment? Issues can arise when the Python packages for the OS clash with the requirements of the tool.

4) Can you use the provided Dockerfile and documentation (https://github.com/opsdisk/metagoofil#docker-installation--usage) to run it in a container?

[Codeiology]

Hey @opsdisk !

1. Yes, I am cloning the repo and running metagoofil.py
2. The exact command I was running was python3 metagoofil.py -t pdf -l 50 -d https://www.apple.com (I was doing Apple to test it idk. It just doesn't work with all the sites I've tried)
3. I have tried getting a Kali Linux virtual machine on my Mac ever since I learned Python. I still can't find any tutorials or versions or emulators or virtualizers that don't give me an error. I guess the CPU architecture is always incompatible. Even when I try to get a Kali Linux arm64 version, still none of them work. I tried UTM, and VMware, but VirtualBox only has a developer version for arm64 that doesn't work for me again with my computer.
4. Ok, here is the embarrassing part. I have docker installed with brew install docker, but whenever I run something it says that the docker daemon is not running. I have searched the stack overflow for a way to start it and even asked chatGPT, but couldn't find anything. If you could tell me, that would be incredible!

[opsdisk]

1. Thanks for that info.
2. What doesn't work about it? Any more verbose screenshots would help. I'm trying to figure out if it's the interaction with Google, or when it tries to download the files from the site. Apple may have throttling on requests against their websites, hence the HTTP 429. Try a .edu site instead.
3. Doesn't sound like a metagoofil issue
4. I don't use Mac OS, but this looked promising - https://apple.stackexchange.com/questions/373888/how-do-i-start-the-docker-daemon-on-macos You could try running docker in a Linux VM...it's much more straightforward.

[Codeiology]

1. Always welcome
2. I tried it again with https://www.url.edu and still the same error I screenshotted happened. It throws that error with all the websites I try it on, regardless of the domain type or bot detection level.
3. I am not sure, but I have some code in Python for a google searcher that does work. Maybe I can make a new repo or something. A collaboration?
4. I tried the instructions. I am running on an ARM64 CPU architecture, so the brew install virtualbox throws Error: Cask virtualbox depends on hardware architecture being one of [{:type=>:intel, :bits=>64}], but you are running {:type=>:arm, :bits=>64}..

[opsdisk]

Hey @Codeiology - Where did you end up with this? Trying to figure out if it's a metagoofil / code issue worth keeping the GitHub issue open for.

[Codeiology]

It could just be a combination of the fact that I am running mac and have an ARM64 CPU architecture. I might try making something like metagoofil for mac. Sorry for leaving this issue open for so long.