Open digininja opened 1 year ago
Hey digi, are you still having this issue? I've not seen it yet running on Parrot.
Also out of curiosity, assuming it still happens, does anything change if you bump the geckodriver version from 0.32.0 to the latest (0.34.0)?
[EDIT] Crazy that I've gotten this far in my career without learning how to read... Not confident that the geckodriver version would interact with Selenium, but still curious if this issue persists on your setup.
I've just checked and after an update it was still failing and updating to 0.34.0 it also fails.
Hmm, interesting. Using Selenium 4.9.1? Just checking, I know the installer currently forces that version but wondering if Ubuntu possibly ships with Selenium of a different version.
I'll give that a try when I can. I can't see the setup script from here, is it easy to switch versions?
On Fri, 5 Jan 2024, 18:29 moth, @.***> wrote:
Hmm, interesting. Using Selenium 4.9.1? Just checking, I know the installer currently forces that version but wondering if Ubuntu possibly ships with Selenium of a different version.
— Reply to this email directly, view it on GitHub https://github.com/RedSiege/EyeWitness/issues/631#issuecomment-1879095321, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWIGJWN26S3KG6HUT2LYNBA73AVCNFSM6AAAAAA77TSA72VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZZGA4TKMZSGE . You are receiving this because you authored the thread.Message ID: @.***>
Should be. I'm going to check on an Ubuntu system whether my hunch is correct.
The process would just be to uninstall the existing Selenium and then python3 -m pip install selenium==4.9.1
. If you want to check the current Selenium version, run python3 -c "import selenium; print(selenium.__version__)"
.
As far as I know, Pip will not overwrite an existing package unless forced, and I don't think the setup script explicitly warns about that currently. This is mostly just a sanity check for me before I decide to start tearing into things.
I'll check when I'm back home later or in the morning.
On Fri, 5 Jan 2024, 19:23 moth, @.***> wrote:
Should be. I'm going to check on an Ubuntu system whether my hunch is correct.
The process would just be to uninstall the existing Selenium and then python3 -m pip install selenium==4.9.1. If you want to check the current Selenium version, run python3 -c "import selenium; print(selenium.version)".
As far as I know, Pip will not overwrite an existing package unless forced, and I don't think the setup script explicitly warns about that currently. This is mostly just a sanity check for me before I decide to start tearing into things.
— Reply to this email directly, view it on GitHub https://github.com/RedSiege/EyeWitness/issues/631#issuecomment-1879155972, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWIBU7GJL6JNWWLIAL3YNBHLHAVCNFSM6AAAAAA77TSA72VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQNZZGE2TKOJXGI . You are receiving this because you authored the thread.Message ID: @.***>
I've just checked, there are no system Selenium packages installed:
$ dpkg --get-selections selenium
dpkg: no packages found matching selenium
Trying to install 4.9.1 says it is already installed:
$ sudo python3 -m pip install selenium==4.9.1
[sudo] password for robin:
Requirement already satisfied: selenium==4.9.1 in /usr/local/lib/python3.10/dist-packages (4.9.1)
Requirement already satisfied: urllib3<3,>=1.26 in /usr/lib/python3/dist-packages (from urllib3[socks]<3,>=1.26->selenium==4.9.1) (1.26.5)
Requirement already satisfied: trio~=0.17 in /usr/local/lib/python3.10/dist-packages (from selenium==4.9.1) (0.22.0)
Requirement already satisfied: trio-websocket~=0.9 in /usr/local/lib/python3.10/dist-packages (from selenium==4.9.1) (0.9.2)
Requirement already satisfied: certifi>=2021.10.8 in /usr/local/lib/python3.10/dist-packages (from selenium==4.9.1) (2022.9.24)
Requirement already satisfied: attrs>=19.2.0 in /usr/lib/python3/dist-packages (from trio~=0.17->selenium==4.9.1) (21.2.0)
Requirement already satisfied: sortedcontainers in /usr/lib/python3/dist-packages (from trio~=0.17->selenium==4.9.1) (2.1.0)
Requirement already satisfied: async-generator>=1.9 in /usr/local/lib/python3.10/dist-packages (from trio~=0.17->selenium==4.9.1) (1.10)
Requirement already satisfied: idna in /usr/lib/python3/dist-packages (from trio~=0.17->selenium==4.9.1) (3.3)
Requirement already satisfied: outcome in /usr/local/lib/python3.10/dist-packages (from trio~=0.17->selenium==4.9.1) (1.2.0)
Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from trio~=0.17->selenium==4.9.1) (1.3.0)
Requirement already satisfied: exceptiongroup>=1.0.0rc9 in /usr/local/lib/python3.10/dist-packages (from trio~=0.17->selenium==4.9.1) (1.0.0rc9)
Requirement already satisfied: wsproto>=0.14 in /usr/lib/python3/dist-packages (from trio-websocket~=0.9->selenium==4.9.1) (1.0.0)
Requirement already satisfied: PySocks!=1.5.7,<2.0,>=1.5.6 in /usr/local/lib/python3.10/dist-packages (from urllib3[socks]<3,>=1.26->selenium==4.9.1) (1.7.1)
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
I've uninstalled it:
$ sudo python3 -m pip uninstall selenium==4.9.1
Found existing installation: selenium 4.9.1
Uninstalling selenium-4.9.1:
Would remove:
/usr/local/lib/python3.10/dist-packages/selenium-4.9.1.dist-info/*
/usr/local/lib/python3.10/dist-packages/selenium/*
Proceed (Y/n)? y
Successfully uninstalled selenium-4.9.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
and reinstalled it:
$ sudo python3 -m pip install selenium==4.9.1
Collecting selenium==4.9.1
Downloading selenium-4.9.1-py3-none-any.whl (6.6 MB)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 6.6/6.6 MB 13.8 MB/s eta 0:00:00
Requirement already satisfied: urllib3<3,>=1.26 in /usr/lib/python3/dist-packages (from urllib3[socks]<3,>=1.26->selenium==4.9.1) (1.26.5)
Requirement already satisfied: trio~=0.17 in /usr/local/lib/python3.10/dist-packages (from selenium==4.9.1) (0.22.0)
Requirement already satisfied: trio-websocket~=0.9 in /usr/local/lib/python3.10/dist-packages (from selenium==4.9.1) (0.9.2)
Requirement already satisfied: certifi>=2021.10.8 in /usr/local/lib/python3.10/dist-packages (from selenium==4.9.1) (2022.9.24)
Requirement already satisfied: attrs>=19.2.0 in /usr/lib/python3/dist-packages (from trio~=0.17->selenium==4.9.1) (21.2.0)
Requirement already satisfied: sortedcontainers in /usr/lib/python3/dist-packages (from trio~=0.17->selenium==4.9.1) (2.1.0)
Requirement already satisfied: async-generator>=1.9 in /usr/local/lib/python3.10/dist-packages (from trio~=0.17->selenium==4.9.1) (1.10)
Requirement already satisfied: idna in /usr/lib/python3/dist-packages (from trio~=0.17->selenium==4.9.1) (3.3)
Requirement already satisfied: outcome in /usr/local/lib/python3.10/dist-packages (from trio~=0.17->selenium==4.9.1) (1.2.0)
Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from trio~=0.17->selenium==4.9.1) (1.3.0)
Requirement already satisfied: exceptiongroup>=1.0.0rc9 in /usr/local/lib/python3.10/dist-packages (from trio~=0.17->selenium==4.9.1) (1.0.0rc9)
Requirement already satisfied: wsproto>=0.14 in /usr/lib/python3/dist-packages (from trio-websocket~=0.9->selenium==4.9.1) (1.0.0)
Requirement already satisfied: PySocks!=1.5.7,<2.0,>=1.5.6 in /usr/local/lib/python3.10/dist-packages (from urllib3[socks]<3,>=1.26->selenium==4.9.1) (1.7.1)
Installing collected packages: selenium
Attempting uninstall: selenium
Found existing installation: selenium 4.0.0a1
Uninstalling selenium-4.0.0a1:
Successfully uninstalled selenium-4.0.0a1
Successfully installed selenium-4.9.1
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
And it is still broken.
I've also tried removing the pip versions of the libraries and found I've already got all the apt versions of them installed, however EW can't see netaddr despite it being installed:
$ sudo apt install -y python3-fuzzywuzzy python3-pyvirtualdisplay python3-netaddr python3-levenshtein
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
python3-netaddr is already the newest version (0.8.0-2).
python3-fuzzywuzzy is already the newest version (0.18.0-2).
python3-levenshtein is already the newest version (0.12.2-2build1).
python3-pyvirtualdisplay is already the newest version (2.2-1).
0 to upgrade, 0 to newly install, 0 to remove and 6 not to upgrade.
$ ./EyeWitness.py --single https://digi.ninja
Traceback (most recent call last):
File "/home/robin/tools/web/EyeWitness/Python/./EyeWitness.py", line 13, in <module>
from modules import db_manager
File "/home/robin/tools/web/EyeWitness/Python/modules/db_manager.py", line 4, in <module>
from modules.objects import HTTPTableObject
File "/home/robin/tools/web/EyeWitness/Python/modules/objects.py", line 5, in <module>
from modules.helpers import strip_nonalphanum
File "/home/robin/tools/web/EyeWitness/Python/modules/helpers.py", line 12, in <module>
from netaddr import IPAddress
ModuleNotFoundError: No module named 'netaddr'
Any ideas?
By coincidence, I do a yearly test for a client from a Kali box on their network. This box hasn't been turned on for a year so I've just updated EW and it has ran fine.
There is definitely something different between Kali and Ubuntu.
I'll have to get an Ubuntu machine up to test, I'll report back with what I find. Thanks for the additional information and troubleshooting efforts.
Able to replicate on an Ubuntu 22.04 VM. The fix appears to be the same as in issue #604.
If you would when you have a moment, can you follow the instructions linked in that issue to uninstall the snap store Firefox and install the proper version from repo?
Assuming that fixes the problem, I don't think we should close this issue just yet until we address this in the setup.sh script, as this feels like something we should be accounting for during setup.
Thanks.
Bloody snap, always causing problems.
I'll give it a try later.
On Tue, 9 Jan 2024, 06:37 moth, @.***> wrote:
Able to replicate on an Ubuntu 22.04 VM. The fix appears to be the same as in issue #604 https://github.com/RedSiege/EyeWitness/issues/604.
If you would when you have a moment, can you follow the instructions linked in that issue to uninstall the snap store Firefox and install the proper version from repo?
Assuming that fixes the problem, I don't think we should close this issue just yet until I have a chance to PR my latest setup.sh script stuff, as this feels like something we should be accounting for during setup.
Thanks.
— Reply to this email directly, view it on GitHub https://github.com/RedSiege/EyeWitness/issues/631#issuecomment-1882490142, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWP2TPCUJOA5W33MOZ3YNTQTJAVCNFSM6AAAAAA77TSA72VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBSGQ4TAMJUGI . You are receiving this because you authored the thread.Message ID: @.***>
Worked first time!
Coming up with a fix for this is going to be tricky as forcing people off the snap version just to run this tool doesn't feel like it would go down very well.
Agreed, though right now the solution appears to be either overwriting the snap version automatically or prompting the user to do it manually. I feel like if Ubuntu users are already going to be required to do it manually to use EW, it might not make much difference to do it in the setup, assuming the user is notified beforehand.
Alternatively, maybe we should compare the snap and apt Firefox versions to see if there's a possible fix or workaround so this wouldn't be necessary. Might take a look at that as time allows, though I'm happy enough that a fix exists for the time being.
A quick guess would be that the snap is failing due to sandboxing that you might not be able to get around.
But that could be completely wrong.
On Tue, 9 Jan 2024, 17:48 moth, @.***> wrote:
Agreed, though right now the solution appears to be either overwriting the snap version automatically or prompting the user to do it manually. I feel like if Ubuntu users are already going to be required to do it manually to use EW, it might not make much difference to do it in the setup, assuming the user is notified beforehand.
Alternatively, maybe we should compare the snap and apt Firefox versions to see if there's a possible fix or workaround so this wouldn't be necessary. Might take a look at that as time allows, though I'm happy enough that a fix exists for the time being.
— Reply to this email directly, view it on GitHub https://github.com/RedSiege/EyeWitness/issues/631#issuecomment-1883516456, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWPVPGI775QXVWKVZK3YNV7HVAVCNFSM6AAAAAA77TSA72VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBTGUYTMNBVGY . You are receiving this because you authored the thread.Message ID: @.***>
I think that you're on to something there, but thankfully might be wrong that it's not fixable. Check this out:
https://stackoverflow.com/a/72237794
I just tested it. Doing TMPDIR=./ python3 EyeWitness.py --single http://127.0.0.1:8000
worked.
That's promising.
On Tue, 9 Jan 2024, 17:55 moth, @.***> wrote:
I think that you're on to something there, but thankfully might be wrong that it's not fixable. Check this out:
https://stackoverflow.com/a/72237794
I just tested it. Doing TMPDIR=./ python3 EyeWitness.py --single http://127.0.0.1:8000 worked.
— Reply to this email directly, view it on GitHub https://github.com/RedSiege/EyeWitness/issues/631#issuecomment-1883526267, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWPDOWALCKVC56FZCZDYNWABBAVCNFSM6AAAAAA77TSA72VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBTGUZDMMRWG4 . You are receiving this because you authored the thread.Message ID: @.***>
Ugh, but it still feels more like a workaround than a "fix", doesn't it. Changing the invocation does seem less invasive than installing a different version of Firefox, at least...
Perhaps we can just update the README and/or prompt in the setup.sh script that those are the two options? I'll keep looking and see if there's a good/clean way to manage this.
Can you set the variable in the script before calling out to the apps?
On Tue, 9 Jan 2024, 18:03 moth, @.***> wrote:
Ugh, but it still feels more like a workaround than a "fix", doesn't it. Changing the invocation does seem less invasive than installing a different version of Firefox, at least...
Perhaps we can just update the README and/or prompt in the setup.sh script that those are the two options? I'll keep looking and see if there's a good/clean way to manage this.
— Reply to this email directly, view it on GitHub https://github.com/RedSiege/EyeWitness/issues/631#issuecomment-1883537555, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWPS6YXM3354B2IP6WDYNWA7PAVCNFSM6AAAAAA77TSA72VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBTGUZTONJVGU . You are receiving this because you authored the thread.Message ID: @.***>
That's one of the things I'm considering, yeah. I'm also looking at whether Selenium has a clean way to directly specify the temporary directory.
Setting os.environ["TMPDIR"] = "./"
in EyeWitness.py seems to work.
Perhaps a good solution would be to set the TMPDIR
variable to the EyeWitness.py directory by default and add a new CLI argument so people can overwrite it if they want? I wonder if people would find that agreeable.
You'd just need to pick a location that is more than likely to be writable for most users. You can't pick the current directory or install directory in case the tool was installed as root.
On Tue, 9 Jan 2024, 18:24 moth, @.***> wrote:
Setting os.environ["TMPDIR"] = "./" in EyeWitness.py seems to work.
Perhaps a good solution would be to set the TMPDIR variable to the EyeWitness.py directory by default and add a new CLI argument so people can overwrite it if they want? I wonder if people would find that agreeable.
— Reply to this email directly, view it on GitHub https://github.com/RedSiege/EyeWitness/issues/631#issuecomment-1883566260, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWJRSY7T65GHT2IOZXDYNWDMHAVCNFSM6AAAAAA77TSA72VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBTGU3DMMRWGA . You are receiving this because you authored the thread.Message ID: @.***>
Make sure you clear it out when done do you don't accidentally leave stuff behind that you shouldn't.
On Tue, 9 Jan 2024, 18:24 moth, @.***> wrote:
Setting os.environ["TMPDIR"] = "./" in EyeWitness.py seems to work.
Perhaps a good solution would be to set the TMPDIR variable to the EyeWitness.py directory by default and add a new CLI argument so people can overwrite it if they want? I wonder if people would find that agreeable.
— Reply to this email directly, view it on GitHub https://github.com/RedSiege/EyeWitness/issues/631#issuecomment-1883566260, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWJRSY7T65GHT2IOZXDYNWDMHAVCNFSM6AAAAAA77TSA72VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBTGU3DMMRWGA . You are receiving this because you authored the thread.Message ID: @.***>
Fair, yeah. Just teasing out the solution. I'm also still looking for whether you can just give Snap applications the ability to write to /tmp, though like you said that might not be possible to work around.
I'm happy to try to break anything you come up with.
On Tue, 9 Jan 2024, 18:32 moth, @.***> wrote:
Fair, yeah. Just teasing out the solution. I'm also still looking for whether you can just give Snap applications the ability to write to /tmp, though like you said that might not be possible to work around.
— Reply to this email directly, view it on GitHub https://github.com/RedSiege/EyeWitness/issues/631#issuecomment-1883577462, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWJ2NVE7Y6TTNREKDVTYNWEK3AVCNFSM6AAAAAA77TSA72VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBTGU3TONBWGI . You are receiving this because you authored the thread.Message ID: @.***>
Actually, from what I'm reading, you can't even run snap applications from root. The current user is required to have a home directory under /home
in order to function. So that might be a non-starter right there. I'm wondering if that might be a reason to avoid using the snap Firefox, though it's hard to say how many users will be trying to run EW on Ubuntu under the root user.
I'd say not many and those who are should know better.
On Tue, 9 Jan 2024, 18:41 moth, @.***> wrote:
Actually, from what I'm reading, you can't even run snap applications from root. The current user is required to have a home directory under /home in order to function. So that might be a non-starter right there. I'm wondering if that might be a reason to avoid using the snap Firefox, though it's hard to say how many users will be trying to run EW on Ubuntu under the root user.
— Reply to this email directly, view it on GitHub https://github.com/RedSiege/EyeWitness/issues/631#issuecomment-1883590751, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWNEM24AASBRLCUXXR3YNWFO7AVCNFSM6AAAAAA77TSA72VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQOBTGU4TANZVGE . You are receiving this because you authored the thread.Message ID: @.***>
OS Used - ALL Information (architecture, linux flavor, etc.)
Ubuntu 22.04.3 LTS - fully up to date
Pulled from repo and setup ran.
error you are encountering
When it runs, it sits for ages showing nothing then finally errors with:
Hitting ctrl-c gives this stack trace.
Expected behavior (vs. what you encountered)
Ideally it should take a screenshot.
Any additional information
Silent whispers speak, Eyes bear witness to truth, Moments etched in gaze.