placeholder to get all selected urls !

Hi @0xAwali , sorry for the late response. My solution would be to use a wrapper script like this:

#!/usr/bin/python3
import sys
import re
import argparse
from subprocess import Popen, PIPE, CalledProcessError

def eprint(*args, **kwargs):
    print(*args, file=sys.stderr, **kwargs)

try:
    from urlextract import URLExtract
except:
    eprint("ERROR: urlextract not found! Use the following command to fix this error:")
    eprint("       > pip3 install urlextract")
    sys.exit(1)

parser = argparse.ArgumentParser(description='Extract urls from file and execute user specified program to handle ')
parser.add_argument('-f','--filter', help='Regex filter for urls', required=False)
parser.add_argument('-v','--verbose', action='store_true', help='Verbosely list commands executed.', required=False)
parser.add_argument('-d','--dry-run', action='store_true', help='Parses the file but prints command to execute instead of executing it.')
parser.add_argument('-c','--command', help='The command to execute. Use {{URL}} to specify where extracted URLs should be placed.', required=True)
parser.add_argument('input_file', help='The file to parse.')
args = vars(parser.parse_args())

input_file = args['input_file']
url_filter = re.compile(args['filter']) if 'filter' in args else ''
dry_run = True if args['dry_run'] else False
verbose = True if args['verbose'] else False

with open(input_file) as f: 
    data = " ".join(line.strip() for line in f)
    for url in URLExtract().find_urls(data):
        if verbose:
            eprint('URL: ' + url)
        if url_filter and not url_filter.match(url):
            continue
        command = args['command'].replace('{{URL}}', url)
        if verbose or dry_run:
            eprint('CMD: ' + command)
        if not dry_run:
            with Popen(command, shell=True, stdout=PIPE, bufsize=1, universal_newlines=True) as p:
                for line in p.stdout:
                    print(line, end='')

Example:

input.txt:

urls like http://example.com and http://more.info but also
things like abc.com and
other.stuff.

Execute:

# Process all urls in input.txt and execute 'echo {{URL}}'
python3 urlextract-and-execute.py -c 'echo {{URL}}' input.txt 
# Define a filter to only process certain types of urls
python3 urlextract-and-execute.py -f '^http(s)?://.*' -c 'echo {{URL}}' input.txt

Note: I've also added a -d | --dry-run option. Make sure to use it prior to execution to see what would actually be executed.

I'm closing this issue for now. If you have any questions or remarks just comment below 🙏

bytebutcher / burp-send-to

placeholder to get all selected urls ! #12