search

digininja / CeWL

CeWL is a Custom Word List Generator
1.96k stars 258 forks source link

Not working for javascript rendered HTML pages #53

Closed anandumdas closed 4 years ago

anandumdas commented 4 years ago

The target app uses react js and the html pages are dynamically generated from the front end. CeWl fails to collect words in this case. Any solutions?

digininja commented 4 years ago

It won't work as it does no JavaScript execution.

The only idea I've got for stuff like that is to browse the site through burp then save out the rendered pages and process those. You could either do it on the command line or get it all in one file and through it on a web server and point CeWL at the one page.

On Sun, 10 Nov 2019, 07:26 Anandu M Das, notifications@github.com wrote:

The target app uses react js and the html pages are dynamically generated from the front end. CeWl fails to collect words in this case. Any solutions?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/digininja/CeWL/issues/53?email_source=notifications&email_token=AAA4SWIPE2QTEVONUVQQYETQS6ZRNA5CNFSM4JLLIIH2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HYHGJLQ, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWLVU24E5PPFMWR6BTLQS6ZRNANCNFSM4JLLIIHQ .

anandumdas commented 4 years ago

Thanks for the quick response.

And probably, you can consider it as a new feature, by making use of something like this?

digininja commented 4 years ago

I'll have a look through it and see.

On Sun, 10 Nov 2019, 08:33 Anandu M Das, notifications@github.com wrote:

Closed #53 https://github.com/digininja/CeWL/issues/53.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/digininja/CeWL/issues/53?email_source=notifications&email_token=AAA4SWPL2RIMIJRAO2UHPKDQS7BMRA5CNFSM4JLLIIH2YY3PNVWWK3TUL52HS4DFWZEXG43VMVCXMZLOORHG65DJMZUWGYLUNFXW5KTDN5WW2ZLOORPWSZGOUYGNK6Y#event-2785858939, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWKMA4GCSTCRZ5IRJFTQS7BMRANCNFSM4JLLIIHQ .

sebaspf commented 1 month ago

I solved it using https://www.brow.sh/ . Browsh is a text-based browser that renders the websites (html, css, js, images, etc. ) using an headless Firefox. You can start is in server mode and request a plain text version of the site.

# start the server
browsh --http-server-mode

# Open a new terminal
# Request the site in plain text using a header
 cewl --header "X-Browsh-Raw-Mode: PLAIN" localhost:4333/https://google.com
digininja commented 1 month ago

@sebaspf Nice idea. Fancy submitting a PR to the README file on how to get this working? Would need to be really simple and go from install to fully running.