search

jonluca / Anubis-DB

Database to store previously found subdomains
56 stars 11 forks source link

SW Design Suggestions #1

Closed wilkenstein closed 4 years ago

wilkenstein commented 6 years ago

Wew, will use this issue for this project and a similar one for Anubis.

wilkenstein commented 6 years ago

https://github.com/jonluca/Anubis-DB/blob/master/models/domains.js#L1

It's always tempting to use MongoDB, especially when it comes to JS. Mongo has come a long ways, see for instance https://jepsen.io/analyses/mongodb-3-4-0-rc3. So if you're using the latest and greatest mongo, and you're comfortable with it, it's probably okay.

I've steered clear of mongo in the past because of its many, many flaws. All databases have operational flaws, but mongo used to stand out as one of the worst.

Since this project is already somewhat of a database abstraction layer, it's probably good to stick with one and only one database tech, and use that throughout the code.

wilkenstein commented 6 years ago

Just curious why this is an express server and not, say, a library?

wilkenstein commented 6 years ago

Tying Anubis and the database together so tightly (Anubis queries AnubisDB, Anubis sends out to AnubisDB) is slightly odd. A potentially different approach would be for Anubis to support outputting as CSV or other formats, and an AnubisDB client tool that could suck in that CSV.

IME, security researchers like the data itself, and love some data sources like AnubisDB, but they also like having the data themselves and playing around with different sources, etc.