search

pry0cc / axiom

The dynamic infrastructure framework for everybody! Distribute the workload of many different scanning tools with ease, including nmap, ffuf, masscan, nuclei, meg and many more!
MIT License
4.07k stars 646 forks source link

[Feature] Saving storage and time not duplicating logs and output #774

Open pdelteil opened 11 months ago

pdelteil commented 11 months ago

So, there's this case when some modules create the same data on the logs and output folder. For example, waybackurls, gau, this has this problems:

  1. When the scan is finished the merging process might use all the storage if the output is big.

In a recent scan I ended up with the following:

35 GB ../logs/ 35 GB ../output/ 35 GB outputfile.txt

I had to manually remove the logs and output folders but it failed in previous attempts because of lack of storage.

Solution

  1. In this kind of modules the only data generated should be the /output , locally and remotely.
iambouali commented 10 months ago

Have you considered using the --rm-logs flag while scanning with Axiom? It might be usefull in addressing this issue

iambouali commented 10 months ago

I have experienced this problem several times too ...

pdelteil commented 10 months ago

Have you considered using the --rm-logs flag while scanning with Axiom? It might be usefull in addressing this issue

Doesn't help, because it will remove the logs after the scan, not during the scan.

iambouali commented 10 months ago

gotcha! Yes, in my opinion this improvement would also be very useful