search

DedSecInside / TorBot

Dark Web OSINT Tool
Other
2.94k stars 536 forks source link

Turn link tree into JSON #142

Closed KingAkeem closed 3 years ago

KingAkeem commented 6 years ago

Is your feature request related to a problem? Please describe. We need to be able to insert our trees into a DB and this will be easily done as JSON if we can figure out an algorithm/library/etc that can convert the link tree into a JSON object that can be stored.

Describe the solution you'd like Not sure yet, open to suggestions.

ghost commented 6 years ago

If you already have a linked tree, might consider using a document database to alleviate conversion?

KingAkeem commented 6 years ago

What's a document database? I'm not aware of this concept, it may be a perfect solution.

cardinalion commented 5 years ago

Document-oriented database is one kind of NoSQL db, MongoDB is a common one.

KingAkeem commented 5 years ago

Sounds like it could be useful for this case, would you like to take on the task?

cardinalion commented 5 years ago

I'm not very clear. Do you want to save the trees in mongodb? If so, will the trees be modified frequently?

KingAkeem commented 5 years ago

I think that'd be a pretty cool feature. The trees won't be modified frequently at runtime. I was thinking that we could add an option so that users could store the trees in a DB that could be queried to retrieve the tree. The workflow would be

  1. User inputs a command to generate a tree with a given URL and a flag to insert it into the DB.
  2. After the tree is inserted a. User should be able to query for the tree. b. User should be able to remove the tree. c. User should be able to visualize a tree that exist within the DB d. User should be able to download a tree that exist within the DB

Does that make sense?

fepitre commented 5 years ago

I think that'd be a pretty cool feature. The trees won't be modified frequently at runtime. I was thinking that we could add an option so that users could store the trees in a DB that could be queried to retrieve the tree. The workflow would be

1. User inputs a command to generate a tree with a given URL and a flag to insert it into the DB.

2. After the tree is inserted
   a. User should be able to query for the tree.
   b. User should be able to remove the tree.
   c. User should be able to visualize a tree that exist within the DB
   d. User should be able to download a tree that exist within the DB

Does that make sense?

IMHO, it's a good start. I'm currently working on a prototype but the achievements of your points 2 are currently done using a separate tool. I'm not sure if it should be integrated or not in the main script.

KingAkeem commented 5 years ago

The options (query, remove, visualize, download) should be integrated into the main function because we'll need to be read the arguments from the user.