mac下使用-b貌似没效果

sevck commented 1 year ago

command: url -s all -m 2 -b /api/ -u http://a.com url -s all -m 2 -b api -u http://a.com 都不行

扫描出来的结果是： http://a.com/dashboard/risk_user_line [ Status: 404, Size: 842, Title: 404 Not Found ]

实际API路径为 http://a.com/api/dashboard/risk_user_line

pingc0y commented 1 year ago

下次更新修复，上次更新稍微改了下结构，出现了好几个bug呀 0.0

pingc0y commented 1 year ago

已更新

sevck commented 1 year ago

更新后试了好，遇到比较诡异的问题.. 挂上代理，代理是burp，看只发了一个请求 [sevck@MacBook-Pro ~ ]$ url -s all -m 2 -u http://xxx.cn -x http://192.168.220.2:8081

/\ /\ /\ / / / () | | / / \ \/ \/// / / \ | | ' \ / ` |/ \ '| \ _/ / \ / / | | | | | (| | / | \/\/ __\/ ||| ||_,|_|_|

By: pingc0y UpdateTime: 2023/2/21 Github: https://github.com/pingc0y/URLFinder

Start Spider URL: http://xxxx.cn Spider OK Start 137 Validate... Validate OK 0 JS to xxx.cn

0 JS to Other

0 URL to xxx

0 URL to Other

0 Domain

去掉代理后， [sevck@MacBook-Pro ~ ]$ url -s all -m 2 -u http://xxx.cn

/\ /\ /\ / / / () | | / / \ \/ \/// / / \ | | ' \ / ` |/ \ '| \ _/ / \ / / | | | | | (| | / | \/\/ __\/ ||| ||_,|_|_|

By: pingc0y UpdateTime: 2023/2/21 Github: https://github.com/pingc0y/URLFinder

Start Spider URL: http://xxx.cn Spider OK Start 137 Validate... Validate OK 4 JS to xxx http://xxx.cn/static/js/manifest.1cf26e7104397675e590.js [ Status: 200, Size: 1403 ] http://xxx.cn/static/js/vendor.cb24b37053185ce180a8.js [ Status: 200, Size: 2982103 ] http://xxx.cn/static/js/app.0d3dffa218be111ae255.js [ Status: 200, Size: 2464777 ] https://xxx.cn/water/bat.js [ Status: 200, Size: 9120 ]

3 JS to Other http://timdream.org/wordcloud2.js [ Status: 200, Size: 0 ] https://github.com/wout/svg.filter.js [ Status: 200, Size: 0 ] https://github.com/svgdotjs/svg.draggable.js [ Status: 200, Size: 223619 ]

118 URL to xxx http://xxx.cn/6zp [ Status: 404, Size: 821, Title: 404 Not Found ] http://xxx.cn/O2i [ Status: 404, Size: 821, Title: 404 Not Found ] http://xxx.cn/Q7J [ Status: 404, Size: 821, Title: 404 Not Found ] http://xxx/auth/login [ Status: 404, Size: 828, Title: 404 Not Found ]

sevck commented 1 year ago

不用代理的情况下,添加 -b api:

[sevck@MacBook-Pro ~ ]$ url -s all -m 2 -u http://xxx.cn -b api

/\ /\ /\ / / / () | | / / \ \/ \/// / / \ | | ' \ / ` |/ \ '| \ _/ / \ / / | | | | | (| | / | \/\/ __\/ ||| ||_,|_|_|

By: pingc0y UpdateTime: 2023/2/21 Github: https://github.com/pingc0y/URLFinder

Start Spider URL: http://xxx.cn Spider OK Start 154 Validate... Validate OK 4 JS to xxx http://xxx.cn/static/js/manifest.1cf26e7104397675e590.js [ Status: 200, Size: 1403 ] http://xxx.cn/static/js/vendor.cb24b37053185ce180a8.js [ Status: 200, Size: 2982103 ] http://xxx.cn/static/js/app.0d3dffa218be111ae255.js [ Status: 200, Size: 2464777 ] https://xxx.cn/water/bat.js [ Status: 200, Size: 9120 ]

1 URL to xxx http://xxx.cn/suborder/approve/ [ Status: 200, Size: 647, Title:xxx- OPWF ]

53 URL to Other http://svgjs.dev [ Status: 200, Size: 173 ] https://github.githubassets.com [ Status: 200, Size: 8 ] https://skills.github.com/ [ Status: 200, Size: 13246, Title: GitHub Skills ] https://education.github.com [ Status: 200, Size: 57616, Title: Engaged students are the result of using real-world tools - GitHub Education ] https://docs.github.com/articles/which-remote-url-should-i-use [ Status: 200, Size: 193701, Title: About remote repositories - GitHub Docs ] https://cli.github.com [ Status: 200, Size: 32472, Title: GitHub CLI | Take GitHub to the command line ] https://desktop.github.com [ Status: 200, Size: 11234, Title: GitHub Desktop | Simple collaboration from your desktop ] https://desktop.github.com/ [ Status: 200, Size: 11234, Title: GitHub Desktop | Simple collaboration from your desktop ] https://developer.apple.com/xcode/ [ Status: 200, Size: 44450, Title: Xcode 14 Overview - Apple Developer ] https://svgdotjs.github.io/ [ Status: 200, Size: 173 ] https://docs.github.com/site-policy/privacy-policies/github-privacy-statement [ Status: 200, Size: 327072, Title: GitHub Privacy Statement - GitHub Docs ] https://www.githubstatus.com/ [ Status: 200, Size: 107704, Title: GitHub Status ] https://support.github.com?tags=dotcom-footer [ Status: 200, Size: 71666, Title: GitHub Support ] https://svgdotjs.github.io [ Status: 200, Size: 173 ] https://github-cloud.s3.amazonaws.com [ Status: 403, Size: 243 ] https://user-images.githubusercontent.com/ [ Status: 403, Size: 243 ] https://collector.github.com/github/collect [ Status: 404, Size: 1253, Title: Collector ] https://api.github.com/_private/browser/stats [ Status: 404, Size: 84 ] https://api.github.com/_private/browser/errors [ Status: 404, Size: 84 ] https://api.github.com/_private/browser/optimizely_client/errors [ Status: 404, Size: 84 ] https://resources.github.com/devops/ [ Status: timeout, Size: 0 ] https://resources.github.com/devops/fundamentals/devsecops/ [ Status: timeout, Size: 0 ] https://resources.github.com/ [ Status: timeout, Size: 0 ] http://schema.org/SoftwareSourceCode [ Status: timeout, Size: 0 ] https://github.com/notifications/beta/shelf [ Status: timeout, Size: 0 ] https://github.com/svgdotjs/svg.draggable.js.git [ Status: timeout, Size: 0 ] https://github.com/contact [ Status: timeout, Size: 0 ] https://docs.github.com [ Status: timeout, Size: 0 ] https://github.com [ Status: timeout, Size: 0 ] https://github.githubassets.com/ [ Status: timeout, Size: 0 ] https://github.com/login [ Status: timeout, Size: 0 ] https://docs.github.com/site-policy/github-terms/github-terms-of-service [ Status: timeout, Size: 0 ] https://github.com/security [ Status: timeout, Size: 0 ] https://avatars.githubusercontent.com [ Status: timeout, Size: 0 ] https://github.com/pricing [ Status: timeout, Size: 0 ] https://services.github.com [ Status: timeout, Size: 0 ] https://github.com/about [ Status: timeout, Size: 0 ] https://github.com/svgdotjs/svg.filter.js.git [ Status: timeout, Size: 0 ] https://github.com/ [ Status: timeout, Size: 0 ] https://en.wikipedia.org/wiki/Method_chaining [ Status: timeout, Size: 0 ] https://github.com/Fuzzyma [ Status: timeout, Size: 0 ] https://github.com/hzrd149 [ Status: timeout, Size: 0 ] https://github.com/wout [ Status: timeout, Size: 0 ] https://github.com/mcrossley [ Status: timeout, Size: 0 ] https://github.com/apps/dependabot [ Status: timeout, Size: 0 ] https://github.com/0x-jerry [ Status: timeout, Size: 0 ] https://github.com/luismarcelino [ Status: timeout, Size: 0 ] https://github.com/AidasK [ Status: timeout, Size: 0 ] https://github.com/be-ndee [ Status: timeout, Size: 0 ] https://github.com/keenanamigos [ Status: timeout, Size: 0 ] https://github.com/svgdotjs/svg.draggable.js/tree/ [ Status: timeout, Size: 0 ] https://github.com/svgdotjs/svg.filter.js/tree/ [ Status: timeout, Size: 0 ] https://github.com/application/json [ Status: timeout, Size: 0 ]

不使用代理，不添加-b: [sevck@MacBook-Pro ~ ]$ url -s all -m 2 -u http://xxx.cn

/\ /\ /\ / / / () | | / / \ \/ \/// / / \ | | ' \ / ` |/ \ '| \ _/ / \ / / | | | | | (| | / | \/\/ __\/ ||| ||_,|_|_|

By: pingc0y UpdateTime: 2023/2/21 Github: https://github.com/pingc0y/URLFinder

Start Spider URL: http://xxx.cn Spider OK Start 162 Validate... Validate OK 4 JS to http://xxx.cn/static/js/manifest.1cf26e7104397675e590.js [ Status: 200, Size: 1403 ] http://xxx.cn/static/js/vendor.cb24b37053185ce180a8.js [ Status: 200, Size: 2982103 ] http://xxx.cn/static/js/app.0d3dffa218be111ae255.js [ Status: 200, Size: 2464777 ] https://xxx.cn/water/bat.js [ Status: 200, Size: 9120 ]

4 JS to Other http://timdream.org/wordcloud2.js [ Status: 200, Size: 8803 ] https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/2.2.2/bootstrap.min.js [ Status: 200, Size: 31596 ] https://wordcloud2-js.timdream.org/src/wordcloud2.js [ Status: 200, Size: 37581 ] https://wordcloud2-js.timdream.org/index.js [ Status: 200, Size: 19425 ]

118 URL to xxx http://xxx.cn/risk/risk_timeline [ Status: 404, Size: 836, Title: 404 Not Found ] http://xxx.cn/risk/risk_userinfo [ Status: 404, Size: 836, Title: 404 Not Found ] http://xxx.cn/risk/risk_rule_list [ Status: 404, Size: 837, Title: 404 Not Found ] http://xxx.cn/risk/risk_hot_rule [ Status: 404, Size: 836, Title: 404 Not Found ] http://xxx/userrisk/user_risk_info [ Status: 404, Size: 841, Title: 404 Not Found ] http://xxx.cn/userrisk/user_risk_class [ Status: 404, Size: 842, Title: 404 Not Found ]

理论上如果添加-b api 的效果，应该是会列出，还是我使用-b的方式有错误 http://xxx.cn/api/userrisk/user_risk_class [ Status: 200, Size: 842, Title: User Login]

pingc0y commented 1 year ago

baseurl是个完整的路径，这样设置-b http://xxx.cn/api/ 这个参数执行的条件是这样的假如抓到的结果是 /userrisk/user_risk_class 或者userrisk/user_risk_class，是个不完整的链接

没有使用-b参数的情况，会用域名 http://xxx.cn 直接去拼接结果： http://xxx.cn/userrisk/user_risk_class

使用了-b http://xxx.cn/api/ 参数结果：http://xxx.cn/api/userrisk/user_risk_class

如果抓取的结果，直接就是 http://xxx.cn/userrisk/user_risk_class 那么默认这个链接是完整的，不会去用baseurl路径拼接你这个情况，应该就是抓到就是完整链接了，可以通过导出查看抓取来源，去验证下

至于代理确实还有点问题，我已经重新打包上传，你重新下载就可以了

sevck commented 1 year ago

感谢，这样使用确实可以了

pingc0y / URLFinder

mac下使用-b貌似没效果 #48