Crash on AttributeError or LocationValueError

[fishouttah20]

When scanning ecommerce.barclays.com I see two different possible crashes, seemingly at random:

~/pshtt-docker/pshtt$ docker run --rm -it --name pshtt -v $(pwd):/data -e USER_ID=1042 -e GROUP_ID=1042 pshtt/cli --output=april1.csv --debug --timeout=15 --preload-cache=PRELOAD ecommerce.barclays.com
Fetching Chrome preload list from source...
Starting new HTTPS connection (1): chromium.googlesource.com
"GET /chromium/src/net/+/master/http/transport_security_state_static.json?format=TEXT HTTP/1.1" 200 None
Caching preload list at PRELOAD
Fetching Chrome pending preload list...
Starting new HTTPS connection (1): hstspreload.org
"GET /api/v2/pending HTTP/1.1" 200 325150
Pinging http://ecommerce.barclays.com...
Starting new HTTP connection (1): ecommerce.barclays.com
"GET / HTTP/1.1" 302 213
Starting new HTTP connection (1): ecommerce.barclays.com
"GET / HTTP/1.1" 302 193
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/src/app/pshtt/cli.py", line 73, in <module>
    main()
  File "/usr/src/app/pshtt/cli.py", line 54, in main
    results = pshtt.inspect_domains(domains, options)
  File "pshtt/pshtt.py", line 892, in inspect_domains
    results.append(inspect(domain))
  File "pshtt/pshtt.py", line 61, in inspect
    basic_check(domain.http)
  File "pshtt/pshtt.py", line 216, in basic_check
    ultimate_req = ping(endpoint.url, allow_redirects=True, verify=False)
  File "pshtt/pshtt.py", line 131, in ping
    timeout=TIMEOUT
  File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 71, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/api.py", line 57, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 606, in send
    history = [resp for resp in gen] if allow_redirects else []
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 179, in resolve_redirects
    **adapter_kwargs
  File "/usr/local/lib/python2.7/site-packages/requests/sessions.py", line 585, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 370, in send
    conn = self.get_connection(request.url, proxies)
  File "/usr/local/lib/python2.7/site-packages/requests/adapters.py", line 279, in get_connection
    conn = self.poolmanager.connection_from_url(url)
  File "/usr/local/lib/python2.7/site-packages/requests/packages/urllib3/poolmanager.py", line 143, in connection_from_url
    return self.connection_from_host(u.host, port=u.port, scheme=u.scheme)
  File "/usr/local/lib/python2.7/site-packages/requests/packages/urllib3/poolmanager.py", line 114, in connection_from_host
    raise LocationValueError("No host specified.")
requests.packages.urllib3.exceptions.LocationValueError: No host specified.

~/pshtt-docker/pshtt$ docker run --rm -it --name pshtt -v $(pwd):/data -e USER_ID=1042 -e GROUP_ID=1042 pshtt/cli --output=april1.csv --debug --timeout=15 --preload-cache=PRELOAD ecommerce.barclays.com
Fetching Chrome preload list from source...
Starting new HTTPS connection (1): chromium.googlesource.com
"GET /chromium/src/net/+/master/http/transport_security_state_static.json?format=TEXT HTTP/1.1" 200 None
Caching preload list at PRELOAD
Fetching Chrome pending preload list...
Starting new HTTPS connection (1): hstspreload.org
"GET /api/v2/pending HTTP/1.1" 200 325150
Pinging http://ecommerce.barclays.com...
Starting new HTTP connection (1): ecommerce.barclays.com
"GET / HTTP/1.1" 302 193
Starting new HTTP connection (1): ecommerce.barclays.com
"GET / HTTP/1.1" 302 213
Starting new HTTPS connection (1): ecommerce.barcap.com
"GET / HTTP/1.1" 301 243
"GET /online HTTP/1.1" 301 244
"GET /online/ HTTP/1.1" 302 291
Starting new HTTPS connection (1): live.barcap.com
"GET /UAB/S/ecom/logon/1/default?returnUrl=https%3a%2f%2fecommerce.barcap.com%2fonline%2f HTTP/1.1" 200 10250
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/runpy.py", line 174, in _run_module_as_main
    "__main__", fname, loader, pkg_name)
  File "/usr/local/lib/python2.7/runpy.py", line 72, in _run_code
    exec code in run_globals
  File "/usr/src/app/pshtt/cli.py", line 73, in <module>
    main()
  File "/usr/src/app/pshtt/cli.py", line 54, in main
    results = pshtt.inspect_domains(domains, options)
  File "pshtt/pshtt.py", line 892, in inspect_domains
    results.append(inspect(domain))
  File "pshtt/pshtt.py", line 61, in inspect
    basic_check(domain.http)
  File "pshtt/pshtt.py", line 234, in basic_check
    base_immediate = parent_domain_for(subdomain_immediate)
  File "pshtt/pshtt.py", line 736, in parent_domain_for
    return str.join(".", hostname.split(".")[-2:])
AttributeError: 'NoneType' object has no attribute 'split'

[konklone]

Apologies for dropping this, @fishouttah20 - is this still a reproducible issue for you? Is it only when using Docker?

[IanLee1521]

Hi @fishouttah20 -- I don't think this is reproducible anymore (in fact, in the second example, the code has changed quite a bit, and that hostname.split(...) line doesn't exist anymore.

I'm going to go ahead and close this for now, but if you're still having issues, please open a new ticket, and we'll take a look!