Output only URLs

[raphaelsander]

Hi,

At first, congratulations for this project. I am have an issue, maybe by my mistake, but I'd want to send to stdout only URLs, without flags like [url] and [code-200]. Is it possible?

[random-robbie]

2nd for this please :)

[lachokds]

It's a third from me. I've been getting this even when using the --json flag:

{"input":"https://twitter.com","source":"body","type":"form","output":"https://twitter.com/search/realtime?lang=zh-Hant","status":
0,"length":0}
{"input":"https://twitter.com","source":"body","type":"form","output":"https://twitter.com/search/realtime","status":0,"length":0}
{"input":"https://twitter.com","source":"body","type":"url","output":"https://twitter.com/?lang=sk","status":200,"length":151}
[url] - [code-200] - https://abs.twimg.com/responsive-web/client-web-legacy/main.6de340c9.js
{"input":"https://twitter.com","source":"body","type":"form","output":"https://twitter.com/*?lang=en-GB","status":0,"length":0}
{"input":"https://twitter.com","source":"body","type":"form","output":"https://twitter.com/*?lang=zh-Hant","status":0,"length":0}
{"input":"https://twitter.com","source":"body","type":"form","output":"https://twitter.com/*?ref_src=","status":0,"length":0}
{"input":"https://twitter.com","source":"body","type":"url","output":"https://twitter.com/?lang=pt","status":400,"length":83}
{"input":"https://twitter.com","source":"body","type":"url","output":"https://twitter.com/?lang=sv","status":200,"length":151}

Thanks!

[AncientAutumn]

You can try to pipe them into grep -Eo "(http|https)://[a-zA-Z0-9./?=_%:-]*"

[vrechson]

I submitted this PR to fix this issue for both json and quiet flags: https://github.com/jaeles-project/gospider/pull/69