search

calamity-inc / Soup

The everything library for C++ 17 and beyond with bindings for other languages.
MIT License
15 stars 5 forks source link

"Is IP address a tor exit node?" lookup #14

Open Sainan opened 2 years ago

Sainan commented 2 years ago

https://onionoo.torproject.org/details?search=flag:exit

alessandromrc commented 2 years ago

Could this database be useful for this task?

Sainan commented 2 years ago

I don't want to rely on companies like IP2Location. The link provided in the issue description is the actual source from the Tor Project itself. The reason I haven't implemented this in Soup is because I have no need for it right now so it would only be an "interesting challenge" kinda thing for another day.

alessandromrc commented 2 years ago

I don't want to rely on companies like IP2Location. The link provided in the issue description is the actual source from the Tor Project itself.

I felt that, on my github you could find a repository called tor-data, it has some more restricted infos already extracted in json format

Sainan commented 2 years ago

No offense, but parsing the JSON to limit it to a few fields is nothing special. My main concern with all this netIntel stuff is how I want it to work in a more long-term configuration, because data would need to be updated at some point. It also needs to be stored in memory efficiently as it can get quite big.

alessandromrc commented 2 years ago

No offense, but parsing the JSON to limit it to a few fields is nothing special. My main concern with all this netIntel stuff is how I want it to work in a more long-term configuration, because data would need to be updated at some point. It also needs to be stored in memory efficiently as it can get quite big.

Yeah, the main problem is about keeping it efficient as it is really a LOT of data as you said

Ps. Isn't it possible to make a separated repository to keep all the networking databases related stuff with a script that automatically updates everything or you would prefer to have them built into Soup?

Sainan commented 2 years ago

Ps. Isn't it possible to make a separated repository to keep all the networking databases related stuff with a script that automatically updates everything or you would prefer to have them built into Soup?

It really depends on what the actual use case is, but ideally it would be something auto-updating and efficient.

alessandromrc commented 2 years ago

If you check the repository from mines "tor-data" that I updated yesterday it automatically updates itself every 12h with all the tor nodes and ASNs, wouldn't it be easier to make a script that get's all the data from the various sources like Mullvad, NordVPN and others and then just use the output of it for Soup?

Sainan commented 2 years ago

Yes, that would likely be easier, but simplicity is not a concern.