search

Arachni / arachni

Web Application Security Scanner Framework
http://www.arachni-scanner.com
Other
3.75k stars 757 forks source link

PhantomJS Spawn failure #798

Closed Joeyn414 closed 7 years ago

Joeyn414 commented 7 years ago

I seem to be getting a PhantomJS process spawn failure. Any idea where I could have an issue? I am installing it through gem on a Centos 7.

[] Initializing... [] Preparing plugins... [] ... done. [] BrowserCluster: Initializing 6 browsers... [!] [browser#spawn_phantomjs:1277] BrowserCluster Worker#24913800: Spawning PhantomJS... [!] [browser#spawn_phantomjs:1290] BrowserCluster Worker#24913800: Attempt #0, chose port number 33944 [!] [browser#spawn_phantomjs:1294] BrowserCluster Worker#24913800: Spawning process: /tools/nodejs/node-v0.12.5-linux-x64/bin/phantomjs [!] [browser#spawn_phantomjs:1315] BrowserCluster Worker#24913800: Process spawned, waiting for it to boot-up... [!] [browser#spawn_phantomjs:1333] BrowserCluster Worker#24913800: Spawn timed-out. [!] [browser#spawn_phantomjs:1337] BrowserCluster Worker#24913800: /usr/local/share/gems/gems/arachni-1.4/lib/arachni/processes/executables/base.rb:3:in load': undefined class/module Set (ArgumentError) from /usr/local/share/gems/gems/arachni-1.4/lib/arachni/processes/executables/base.rb:3:in

'

Zapotek commented 7 years ago

Please use the packages, they're there to solve these kinds of problems. Is there a reason you opted for the gem?

frameloss commented 7 years ago

Hi there, also seeing the same problem with an install from the pentesters-framework (not your issue, theirs.) I will just move to using the packages, but to answer the question as why that isn't necessarily optimal ... I use Arachni as part of some continuous integration tests, and automatically rebuild the containers it runs from weekly.

It's just more convenient in the case of ptf to be able to grab the latest version. It's not particularly hard to parse out the downloads page on github, just not the path of least resistance. Anyways, thanks for the great tool.

Zapotek commented 7 years ago

@frameloss It's only convenient so long as it works, in a lot of cases it won't though, which is why the best approach is the packages. You're welcome btw.

Joeyn414 commented 7 years ago

I was in a similar situation as @frameloss but in my case I was using the v1.4 arachni gem with gauntlt so at the time it was easier to install arachni via a gem inside my centos docker container. I have since tested and installed arachni from the packages here. I then set the path in centos for arachni to the bin folder thanks to the help from @Zapotek.

I intend to update my centos docker container with a working copy of gauntlt and arachni soon if anyone is interested in using it.

Here are the steps I performed to download and then set the path for the centos 7 install:

wget https://github.com/Arachni/arachni/releases/download/v1.4/arachni-1.4-0.5.10-linux-x86_64.tar.gz tar -xvf arachni-1.4-0.5.10-linux-x86_64.tar.gz echo 'pathmunge /arachni-1.4-0.5.10/bin' > /etc/profile.d/arachni.sh chmod +x /etc/profile.d/arachni.sh

I forgot to mention, more importantly, this did not solve my phantomjs issue. I know it has something to do with the method I am running it. I am running it inside jenkins with a centos 7 container, I will continue to further troubleshoot but I dont think this issue is being caused by something inside arachni, its something to do with other components on the docker container.

Zapotek commented 7 years ago

The packages bundle their own PhantomJS binary within their own environment. I'm not sure how Jenkins would interfere with that, does the user that runs Arachni have enough permissions to write to the package dir?

I can't think of what could be causing this but since it's not an Arachni issue I'm closing this; feel free to discuss further if you find the solution or something interesting.

Joeyn414 commented 7 years ago

Well I did fix the issue I was having, for some reason the timeout wasn't processing correctly. I see the default was 25 seconds but I know the phantomjs job was failing before that. I included the arachni scan option --browser-cluster-job-timeout 180

Works like a charm now! Thanks again for creating this!

Zapotek commented 7 years ago

I think I know what's going on. The first time the browsers start they build a font cache which can take a while depending on system resources. If you come across this issue again the nightlies allow more time for the spawn and include a bunch of other fixes and optimizations that are worth checking out -- and I could use some more testers.

Btw, the timeout option you set doesn't apply to the spawn, the browsers just started faster the second time around.

Cheers

Joeyn414 commented 7 years ago

Oh OK, I didn't realize it the fonts too so long to load. The nature of docker with Jenkins is that cache cannot be assumed to be there. I would be happy to be a tester for the nightlies, I'll put together an automated build process to start doing validation testing with your builds and my servers.

Once this is done I can publish the feedback here.

Zapotek commented 7 years ago

It may also be a good idea to pre-build the cache. After placing the the package in the docker image, run:

./bin/arachni --browser-cluster-pool-size=1 --scope-page-limit=0 http://blah.com