Unexpected errors in `AwsInstance` and `AzureInstance`

shreyamalviya commented 3 years ago

I'm getting the following errors during environment collection...

... in AwsInstance:

File "/home/shreya/Desktop/Projects/monkey/monkey/infection_monkey/system_info/collectors/environment_collector.py", line 1, in <module>
    from common.cloud.all_instances import get_all_cloud_instances
  File "/home/shreya/Desktop/Projects/monkey/monkey/common/cloud/all_instances.py", line 8, in <module>
    all_cloud_instances = [AwsInstance(), AzureInstance(), GcpInstance()]
  File "/home/shreya/Desktop/Projects/monkey/monkey/common/cloud/aws/aws_instance.py", line 49, in __init__
    AWS_LATEST_METADATA_URI_PREFIX + 'dynamic/instance-identity/document', timeout=2).read().decode())
  File "/home/shreya/Desktop/Projects/monkey/monkey/common/cloud/aws/aws_instance.py", line 81, in _extract_account_id
    return json.loads(instance_identity_document_response)[ACCOUNT_ID_KEY]
  File "/usr/lib/python3.7/json/__init__.py", line 348, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python3.7/json/decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python3.7/json/decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

which arises when the account ID is being extracted in _extract_account_id() because

urllib.request.urlopen(
    AWS_LATEST_METADATA_URI_PREFIX + 'dynamic/instance-identity/document', timeout=2).read().decode())

returns:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type" />
<meta content="no-cache" http-equiv="Pragma" />
<title>Waiting...</title>
<script type="text/javascript">
var pageName = '/';
top.location.replace(pageName);
</script>
</head>
<body> </body>
</html>

... and in AzureInstance:

File "/home/shreya/Desktop/Projects/monkey/monkey/infection_monkey/system_info/collectors/environment_collector.py", line 1, in <module>
    from common.cloud.all_instances import get_all_cloud_instances
  File "/home/shreya/Desktop/Projects/monkey/monkey/common/cloud/all_instances.py", line 8, in <module>
    all_cloud_instances = [AwsInstance(), AzureInstance(), GcpInstance()]
  File "/home/shreya/Desktop/Projects/monkey/monkey/common/cloud/azure/azure_instance.py", line 45, in __init__
    self.try_parse_response(response)
  File "/home/shreya/Desktop/Projects/monkey/monkey/common/cloud/azure/azure_instance.py", line 54, in try_parse_response
    response_data = response.json()
  File "/usr/local/lib/python3.7/dist-packages/requests/models.py", line 898, in json
    return complexjson.loads(self.text, **kwargs)
  File "/home/shreya/.local/lib/python3.7/site-packages/simplejson/__init__.py", line 525, in loads
    return _default_decoder.decode(s)
  File "/home/shreya/.local/lib/python3.7/site-packages/simplejson/decoder.py", line 370, in decode
    obj, end = self.raw_decode(s)
  File "/home/shreya/.local/lib/python3.7/site-packages/simplejson/decoder.py", line 400, in raw_decode
    return self.scan_once(s, idx=_w(s, idx).end())
simplejson.errors.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

which arises when the response is parsed here because

response = requests.get(AZURE_METADATA_SERVICE_URL,
                        headers={"Metadata": "true"},
                        timeout=SHORT_REQUEST_TIMEOUT)

returns:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type" />
<meta content="no-cache" http-equiv="Pragma" />
<title>Waiting...</title>
<script type="text/javascript">
var pageName = '/';
top.location.replace(pageName);
</script>
</head>
<body> </body>
</html>

I'm not sure why this suddenly came up but we should probably catch the exceptions.

VakarisZ commented 3 years ago

@shreyamalviya maybe the AWS or Azure API changed? In that case we have bigger problems than decoding errors, we need to validate that these collectors still actually work

VakarisZ commented 3 years ago

Can you elaborate more on how you're getting this? OS, on cloud or on prem?

shreyamalviya commented 3 years ago

Probably not because I don't always get these errors. I've only ran into these 2-3 times, not sure why. Linux, on prem.

guardicore / monkey

Unexpected errors in `AwsInstance` and `AzureInstance` #902