search

pry0cc / axiom

The dynamic infrastructure framework for everybody! Distribute the workload of many different scanning tools with ease, including nmap, ffuf, masscan, nuclei, meg and many more!
MIT License
4k stars 622 forks source link

[feature] Upload nuclei templates from local folder #471

Closed pdelteil closed 2 years ago

pdelteil commented 2 years ago

Currently the nuclei module uploads a local template to all instances using the -wL flag. It would be helpful if the flag would also work with a folder containing templates.

ResistanceIsUseless commented 2 years ago

@pdelteil I've been playing with ideas and started working on a way to copy config files and templates to all fleets. It's built around axiom-drive thats being tested but maybe we can tweak that and create axiom-configs which helps manage configs for each tool.

@0xtavian what do you think?

0xtavian commented 2 years ago

Currently the nuclei module uploads a local template to all instances using the -wL flag. It would be helpful if the flag would also work with a folder containing templates.

With axiom-scan we treat nuclei templates as wordlist (-w flag), more precisely we treat a remote path to a wordlist (file) or folder the same, by just interpolating the user provided argument, whether thats a wordlist or a folder path, and just passing it to the Nuclei binary. I'm not sure how much I want to keep extending this functionality to things that arent wordlists, such as a folder of nuclei templates. It can be confusing to the end user.

alternatively, axiom-scp supports transferring a local folder to all instances, so its just one extra command to run prior to axiom-scan. Instead of extending the -wL flag, maybe we should add an --axiom-scp option to axiom-scan that would allow arbitrary uploads/downloads of files/folders before/after the scan runs. This will take some extra thought and time to plan out, in the meantime I think we'll stick to having users run the axiom-scp prior to running axiom-scan.

we can leave this issue open in the meantime until we have a better solution.

@pdelteil I've been playing with ideas and started working on a way to copy config files and templates to all fleets. It's built around axiom-drive thats being tested but maybe we can tweak that and create axiom-configs which helps manage configs for each tool.

@0xtavian what do you think?

Possibly. I'd love to see axiom-drive get ported to one more provider, since axiom-drive only works on azure ( and I havent thoroughly tested it on azure yet ), if someone pushes support for one of the other three providers we support (DO,IBM, or Linode), I'll try to do whichever two remain.

ResistanceIsUseless commented 2 years ago

I'm thinking we have something completely separate from the drive functionality. We could use github actions to grab the updated base configs for each tool and save them to configs. I'm thinking there should be two flows for how the configs get added to the images/fleets.

  1. During image creation the configs get copied into the correct places.
  2. Using axiom-configs you can push one or multiple configs to fleets. It will know the default for each config.axiom-configs -nuclei-template template.yaml or axiom-configs -rustscan rustscan.toml or axiom-configs -list updated-paths.list and so on.

The only missing piece is handling the config values like API tokens. It would be nice if there was a way to store those and update each config on creation.

0xtavian commented 2 years ago

So thats what i was planning with SecConfigs https://github.com/0xtavian/SecConfigs.

One way it could work is: The user copies the actions they want to a private repo ( https://github.com/projectdiscovery/nuclei-action and https://github.com/projectdiscovery/nuclei-action for example)

they fill out parts of the action with their API keys, they only should have to do this once.

The action runs, downloads all the tools and creates the final config files.

The user clones their private repo with filled out config files to ~/.axiom/configs

[some logic as you state to process the config ad-hoc], which the --axiom-scp feature of axiom-scan should asset with.

But i do like your idea of axiom-configs.. something we should definitely think about..

0xtavian commented 2 years ago

https://github.com/pry0cc/axiom/issues/531

Duplicate issue. Just posting here for tracking.

@ResistanceIsUseless @pdelteil @iambouali For a fix what do think about adding a flag, When you provide a local path to a directory of templates with -wL and you supply the extra flag it will upload it as a folder instead of a file. It would work similar to -wD/ —distribute-wordlist

0xtavian commented 2 years ago

https://github.com/pry0cc/axiom/blob/%23471/interact/axiom-scan - not completed sold on the -fL flag.. but its simple enough.