search

pry0cc / axiom

The dynamic infrastructure framework for everybody! Distribute the workload of many different scanning tools with ease, including nmap, ffuf, masscan, nuclei, meg and many more!
MIT License
3.91k stars 615 forks source link

[Feature] Saving storage and time not duplicating logs and output #774

Open pdelteil opened 6 months ago

pdelteil commented 6 months ago

So, there's this case when some modules create the same data on the logs and output folder. For example, waybackurls, gau, this has this problems:

  1. When the scan is finished the merging process might use all the storage if the output is big.

In a recent scan I ended up with the following:

35 GB ../logs/ 35 GB ../output/ 35 GB outputfile.txt

I had to manually remove the logs and output folders but it failed in previous attempts because of lack of storage.

Solution

  1. In this kind of modules the only data generated should be the /output , locally and remotely.
iambouali commented 5 months ago

Have you considered using the --rm-logs flag while scanning with Axiom? It might be usefull in addressing this issue

iambouali commented 5 months ago

I have experienced this problem several times too ...

pdelteil commented 5 months ago

Have you considered using the --rm-logs flag while scanning with Axiom? It might be usefull in addressing this issue

Doesn't help, because it will remove the logs after the scan, not during the scan.

iambouali commented 5 months ago

gotcha! Yes, in my opinion this improvement would also be very useful