search

digininja / CeWL

CeWL is a Custom Word List Generator
1.96k stars 258 forks source link

Offsite link #64

Closed manerod closed 4 years ago

manerod commented 4 years ago

Hi i want to use cewl but dont scan anything. CeWL 5.4.8 (Inclusion) Robin Wood (robin@digi.ninja) (https://digi.ninja/) Starting at domain Offsite link, not following: domain Words found i try so much pages but nothing.

digininja commented 4 years ago

?

On Wed, 6 May 2020, 22:21 manerod, notifications@github.com wrote:

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/digininja/CeWL/issues/64, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWN7N53C26ZAOVBUAYLRQHIEJANCNFSM4M2ZHNCQ .

manerod commented 4 years ago

? … On Wed, 6 May 2020, 22:21 manerod, @.***> wrote: — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#64>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWN7N53C26ZAOVBUAYLRQHIEJANCNFSM4M2ZHNCQ .

Edited

digininja commented 4 years ago

You are probably pointing it at a URL which redirects to some other location from the first hit. Check it out in with curl and see what happens or run it in verbose mode.

On Wed, 6 May 2020, 22:23 manerod, notifications@github.com wrote:

? … <#m30406247187306067> On Wed, 6 May 2020, 22:21 manerod, @.***> wrote: — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#64 https://github.com/digininja/CeWL/issues/64>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWN7N53C26ZAOVBUAYLRQHIEJANCNFSM4M2ZHNCQ .

Edited

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/digininja/CeWL/issues/64#issuecomment-624898701, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWIIMRHBZ6NYT2NJ4NLRQHIOLANCNFSM4M2ZHNCQ .

manerod commented 4 years ago

thanks for you reply. Via curl in verbose mode conect well. I try several url and the same thing happens.

digininja commented 4 years ago

Give me an example of one that doesn't work and the output from CeWL running in verbose mode.

On Wed, 6 May 2020, 22:37 manerod, notifications@github.com wrote:

thanks for you reply. Via curl in verbose mode conect well. I try several url and the same thing happens.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/digininja/CeWL/issues/64#issuecomment-624904255, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWPUWBFTUH3575BVP6LRQHKA7ANCNFSM4M2ZHNCQ .

digininja commented 4 years ago

Just so you understand what is going on there, you are requesting the http version of the site but that redirects to https which is considered a different site so won't be spidered. If you want to hit the https version of the site you must explicit specify that as the URL.

On Thu, 28 May 2020, 04:22 manerod, notifications@github.com wrote:

[image: Screenshot_22] https://user-images.githubusercontent.com/51981938/83095276-540f5780-a079-11ea-9b48-a6d69a57988c.png

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/digininja/CeWL/issues/64#issuecomment-635073024, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAA4SWJQKIAJPCLGA7OTPZDRTXKI3ANCNFSM4M2ZHNCQ .