Bypass403

[amiremami]

I'm testing bypass403 module on my own site. I have a question.

For example from cloudflare I added this header rule, so it will work like this:

curl -ks -H 'X-Forwarded-For: 127.0.0.1' -X GET 'https://www.3r.wtf/' -H 'User-Agent: Mozilla/5.0'

Now, I use this:

bbot -t 3r.wtf -m httpx bypass403 -om asset_inventory -c web_spider_distance=4 web_spider_depth=4

Now, I expect bbot to actually bypass it and crawl the links, but it's not crawling.

Can't it automatically add the header to requests to bypass and crawl?

[liquidsec]

I think we could probably solve this by manually emitting the contents of the successful bypass as an http_response, @TheTechromancer do you see any issues with doing that?

@amiremami would you be ok with me testing using your site you posted if we do this?

[amiremami]

Yes of course, it's ok by me 🙏

[TheTechromancer]

It would make sense to add this feature after we've implemented the Web Engine. That will include a helper that returns the response in a JSON format similar to HTTP_RESPONSE.

[liquidsec]

adding on-hold tag