Download HTTP Resources

[clburlison]

This is a pretty common task. Here are some resources that I find useful. Include both bash and python examples.

Check for connection to server + port

Include a snippet why using the ping utility is a bad tool for this.

[clburlison] ~ ○ /usr/bin/nc -z -G 30 github.com 443
Connection to github.com port 443 [tcp/https] succeeded!
[clburlison] ~ ○ echo $?
0
[clburlison] ~ ○ /usr/bin/nc -z -G 30 geeeeeeeethub.com 443
nc: getaddrinfo: nodename nor servname provided, or not known
[clburlison] ~ ○ echo $?
1

Python: Do this with subprocess

Bash: Do this with if/else check

Check for file resource

import stuff here...

    if url.scheme in ['http', 'https', 'file']:
        request = urllib2.Request(urlstring)
        request.get_method = lambda: 'HEAD'
        try:
            response = urllib2.urlopen(request)
        except BaseException as e:
            print 'WARNING: %s URL request failed: %s' % (urlstring, str(e))
    else:
        fail('%s is not a valid URL' % urlstring)

bash? use curl and do stuff here

Check for a Network connection

/usr/sbin/scutil -w State:/Network/Global/DNS -t 180
echo $?
0

python use subprocess and write stuff here