Usage examples: Help needed

[sumgr0]

I've been trying to play around with shadowclones, but need help with commands for below situations:

• Running a single nuclei template on the local machine for all the targets in resolved_subs.txt
• Running multiple nuclei templates on local machine for a single url
• Running subfinder on roots.txt with the providers-config.yaml containing all the api-keys.

@fyoorer, please help.

[sumgr0]

For some reason every time I try running the subfinder commands as below:

python shadowclone.py -i /tmp/roots.txt -o /tmp/SC-subfinder.txt -s 20 -c "/go/bin/subfinder -dL {INPUT}"

and

python shadowclone.py -i /tmp/roots.txt -o /tmp/SC-subfinder.txt -s 20 -c "/go/bin/subfinder -d {INPUT}"

I get this error message everytime:

Encountered a bad command exit code! Command: '/go/bin/subfinder -dL /tmp/infile' Exit code: 1 Stdout: already printed Stderr: already printed

[wall6e]

Hi, check my pull request for subfinder with default providers-config.yaml ! ;)

https://github.com/fyoorer/ShadowClone/pull/37

[sumgr0]

Hi @wall6e,

I've tried the fixes suggested in the PR, but I'm still getting the same error message:

[wall6e]

From my side, I had to delete the runtime image already deployed on AWS before rebuilding it and redeploying it for the changes to take effect.

You can check the generated log file to see the returned error message. It give you an indication about whether the new Dockerfile has been taken into account. Finally, you can also run "ls -al /tmp" to see if the "/tmp/.config" directory has well been created.

[sumgr0]

Ah this worked... Thanks for your help with subfinder @wall6e.

Any ideas about the nuclei scenarios as well...

[sumgr0]

After fixing the Dockerfile from your PR suggestion, I added another line to copy my provider-config.yaml to the image:

COPY provider-config.yaml /tmp/.config/provider-config.yaml

Ensure to copy the file from ~/.config/subfinder/provider-config.yaml to the shadowclones. Since I'm a noob about Dockers, not sure if the file can be copied from the original path itself.

Hope this helps someone :)

[sumgr0]

After a lot of trial and errors... Here is command to run a local template on the Lamdas:

python shadowclone.py -i urls.txt --split 3 --no-split /root/local-template.yaml -o nuclei-output.txt -c "/go/bin/nuclei -l {INPUT} -t {NOSPLIT} -config-directory /tmp/"

This would push your local template to the lamda and run them.

[marcelo321]

@sumgr0 I definitely need a list of examples page, not sure why there isn't one with the popularity this repo has

[fyoorer]

This page has been there since day 1

[sumgr0]

@fyoorer This was more for additional scenarios, the existing ones are sure helpful.

Additionally, with the recent changes there is a constant error with file size being larger than 4mb.

[fyoorer]

That was addressed here too. I don't have releases page where I can point out breaking changes so that's the best I could do, sorry. Other than that, users are free to include any tools they want in the containers so I can't provide example for everything. Once you get a hang of the syntax, it should be easy to create a command for any tool in a similar way.

[fyoorer]

@sumgr0 TL:DR is you need to add data_limit: False to your lithops config to avoid that error

[sumgr0]

I completely agree about users trying out the tools by adding them to the containers.

Suggestion: Maybe have a section in wiki to include different examples that users may share to help others trying it out.

Thanks for the tip on change to the lithops config, it worked. I'm going to run few things off shadowclone once again to speed things up. Cheers!

[marcelo321]

@fyoorer totally bad from my side! I visited the wiki 50 times and didn't see the examples page.

I was clicking on https://github.com/fyoorer/ShadowClone/wiki#see-examples-page-for-actual-syntax-and-commands which redirects nowhere, so I thought there was no examples at all.