Move from APIs to native features implementation?

[naltun]

I ran into a problem the other day, when the 'HTTP Header Grab' feature didn't follow the redirect.

Off the top of my head I can't remember which function did this, but under net/http is a function that will follow redirects up to 10x.

This first got me thinking... And then I received a 'Exhausted API use`, etc. notice today.

The use of APIs is well-done because it allows a user to add an extra layer of security; but even then, I am not aware of HackerTarget's privacy policy. This could be considered a security vulnerability.

Even when pentesting, etc. we should always keep security and privacy of ourselves + our property (machines, etc.) at the forefront of concerns. If we are using HackerTarget's API insecurely, then we're still in trouble.

So considering that most people who would ever touch this program would be considering personal safety, maybe it's just better to implement each feature natively.

I am looking for feedback.

[tweedge]

The use of HackerTarget APIs (and subsequent ratelimiting) will limit this tool's functionality for individuals in broader engagements, or looking to do this at any sort of scale (such as me - so be aware that I'm biased). Having these features implemented natively will solve that, along with providing educational benefit to... basically anyone who interacts with the code in a nontrivial manner. It's a resource cost, yes, but I think justified.

The security and privacy concerns could be solved by implementing some sort of VPN, request proxying, or native Tor integration. Definitely a necessity if you're going to do native implementation. Even if you don't move away from HackerTarget, it's still a 'nice to have' for people who may be interested in their local address not being known by HT.

[naltun]

@tweedge Fantastic feedback man.

Regarding the second set of observations (I 100% on-board for the first half):

The security and privacy concerns could be solved by implementing some sort of VPN, request proxying, or native Tor integration. Definitely a necessity if you're going to do native implementation.

You recommend having something built-in, in addition to preventative steps a user may take?

[naltun]

I've concluded that I prefer native implementations. Even if the public API had unlimited calls, I don't like how I need to rely on it.

As of f481bde I have begun the transition. :raised_hands:

[tweedge]

You recommend having something built-in, in addition to preventative steps a user may take?

Personally speaking, it makes it much more attractive. For people in active engagements, losing a shell, a connection, progress on a scan, etc. because you're messing with network configurations or such things can be a pain. So perhaps I've overstated by saying it's a necessity - all of the aforementioned are inconveniences (and predominantly avoidable inconveniences with a proper setup/adequate knowledge), but it's a good thing to have.

Perhaps that should stay on the backburner a while if you're moving forward with getting away from HT though - that's a daunting enough task as-is!

[naltun]

@tweedge Awesome input man. I'll bear it all in mind, and I think I'll be working with it.

Thanks again.

[naltun]

If it has not already become a bit more obvious, the project is moving to using a native API. Currently I have help from sahitpj. If anyone else is interested in this, find an issue (I'll open up one issue per package soon), and start hacking.

Safe day, software idealists.