search

Isona / dirble

Fast directory scanning and scraping tool
GNU General Public License v3.0
614 stars 87 forks source link

Write output every n seconds #30

Closed theblackturtle closed 5 years ago

theblackturtle commented 5 years ago

Hi, Can it save in n seconds? Thanks.

Isona commented 5 years ago

Would you mind clarifying this a little please, do you mean the output written to the terminal or the output written to files? Also, what sort of use case does this have?

theblackturtle commented 5 years ago

When I run with a domain list and big wordlist, it does not write to output file right there or If I just want to run only 5 minutes, and terminal the program, I don't see the output in the output file. The output file only created when it finish all jobs.

Isona commented 5 years ago

Ah, okay thanks, that might be a reasonable idea as part of a pause/resume scan feature as well, the only issue is that if output was written to the file during the scan, it wouldn't be sorted or indented because the information to do that is only obtained at the end (and it's expensive). It should be possible to write to the file as it goes and then rewrite the file with the sorted information at the end though. I'll look into implementing it.

theblackturtle commented 5 years ago

Ya. You can make it sorted at the end of running because if someone running the program and something is crash or the internet is down, they need to run again. So write output, in the end, is not good in a long time job.

Isona commented 5 years ago

Rather than saving it every N seconds, it now catches ctrl+C and will output everything then, I hope this is a reasonable solution. Otherwise, please reopen the issue and I'm happy to discuss it some more.

theblackturtle commented 5 years ago

I think it should have an option with the write output stream and sorted at the end of scanning. We can see what scanned without waiting for a scanning finish because of it maybe can take a long time. And with the write stream, you can see what things you missing and can improve your command (Add more headers, add User-Agent, Whitelist status code,...) without waiting it finish (Example the scanning take 3-4 hours with huge wordlist). When the scanning finished, the file can be removed and replace with the sorted file.