search

sherlock-project / api

Api for sherlock module
MIT License
63 stars 32 forks source link

First working version of Sherlock API #2

Closed jcs090218 closed 3 years ago

jcs090218 commented 3 years ago

This is the first version of working Sherlock API built with [Django REST Framework]().

Please see my current working fork here.

I have also created a demo web application here.

The working live demo please goto link, http://www.jcs-profile.com/public/sherlock-web/.

Image 1

jcs090218 commented 3 years ago

@sdushantha @TheYahya What do you guys think about this PR? I think the only problem is currently it clones the repository from GitHub instead of downloading through pip. I think this is acceptable since we can easily updated to the master branch by just calling git pull.

Backend

Right now it only takes two endpoints,

  1. /data/ - GET method to get all data from resources/data.json
  2. /cli/ - POST to actually request for sherlock execution

Frontend

The current state of sherlock takes too long per request. I have managed to use --site parameter to get the result in group (current value is 5 result per group) so it will shows information in group. See below,

ezgif-2-2eceda1e2b87

TheYahya commented 3 years ago

This is great @jcs090218, So far seems great job here, but @amirrossein will review your code and will give you specific feedbacks if there was any.

jcs090218 commented 3 years ago

Sounds good to me. Thanks!

jcs090218 commented 3 years ago

Hmm... This has been a while since opened. I think @amirrossein isn't being very active on GitHub? 😕

sdushantha commented 3 years ago

Hi @jcs090218,

Thank you for taking the time and creating this, it looks really nice! I don't anything about Django, so I unfortunately cannot review this code.

But I do want to say that there is a severe security flaw on your website. Because of the way you have made the web version of Sherlock, I can execute shell commands as root on your server. You might want to get this fixed before someone takes advantage of this and completely breaks your website.

To test the vulnerability, just input --help;whoami;uname -a into the input box and you should see that you are root and some information about your server.

I know nothing about web development, but I for sure can say that you should probably sanitize the input to prevent this remote code execution from happening.

jcs090218 commented 3 years ago

Thank you for taking the time and creating this, it looks really nice! I don't anything about Django, so I unfortunately cannot review this code.

Thanks! Glad you like it! :)

I assumed @amirrossein is going to make review on this? You can add me and assign me to the backend or api team so I can respond and fix things quickly. I would definitely like to work with anyone who want to be part of this! :)

But I do want to say that there is a severe security flaw on your website. Because of the way you have made the web version of Sherlock, I can execute shell commands as root on your server. You might want to get this fixed before someone takes advantage of this and completely breaks your website.

I have added the argument check on both client and server sides. TBH, I am not the expert on cyber security and web development. I am very appreciate you feedback on this! :)

Hmm... I wasn't sure if you guys want the frontend? I assumed this PR is only for the backend/api. If you guys want, I can transfer my frontend repository to this organization. It will be great if more people notice the repository (frontend) since I am not very good at web design especially on the UI.

jcs090218 commented 3 years ago

@sdushantha @amirrossein Any progress on this PR? :)

sdushantha commented 3 years ago

@TheYahya It looks like @amirrossein hasn't reviewed this PR. Since you have some experience with web development, could you review it in your spare time?

jcs090218 commented 3 years ago

@TheYahya A friendly ping here. :)

jcs090218 commented 3 years ago

@sdushantha @TheYahya @amirrossein This seems to be a while, hope I'm not being rush here. Let me know if I could help the review process!

sdushantha commented 3 years ago

@jcs090218 I am really sorry about the delay. I'm not sure where the others have gone.

I am currently really busy preparing for my final exams, so I cant take a good at this PR right now, but once I'm done with my exams I'll do my best to review this PR with my limited knowledge on Django.

jcs090218 commented 3 years ago

It's all good! I am not the expert on DRF too! Most of the files are generated by DRF, and I had only made 2 endpoints. I feel great to contribute to such an amazing project! :)

jcs090218 commented 3 years ago

Hi, @sdushantha! Hope you are doing well!

I realized every time a user input a search will generate a .txt file. Is there a way to avoid this? Or else I would have a bunch of output/result files on my server. Thanks!

sdushantha commented 3 years ago

Hi @jcs090218,

Sherlock currently doesn't have an option to prevent the files from generating. I think one thing you could do is to run a script or command after Sherlock has been executed which deletes all files ending with .txt except requirement.txt incase you need that. If it is not needed, just run rm *.txt to delete all files ending with .txt

ccdd13 commented 3 years ago

Hi @jcs090218,

Sherlock currently doesn't have an option to prevent the files from generating. I think one thing you could do is to run a script or command after Sherlock has been executed which deletes all files ending with .txt except requirement.txt incase you need that. If it is not needed, just run rm *.txt to delete all files ending with .txt

or mkdir -p searchs && python sherlock/sherlock.py --folderoutput ./searchs && rm -rf ./searchs also +1 for --no-output argument

jcs090218 commented 3 years ago

@sdushantha Any progress on this? :)

sdushantha commented 3 years ago

I'll merge this PR and we'll see how things go. But from what I see, I think everything should be fine.

jcs090218 commented 3 years ago

You might want to test it on your machine? I have only tested on Windows and Linux. You can try my client if you want to, https://github.com/jcs-workspace/sherlock-web!