google / cloud-forensics-utils

Python library to carry out DFIR analysis on the Cloud
Apache License 2.0
460 stars 88 forks source link

GCP log querying broken #141

Closed hacktobeer closed 4 years ago

hacktobeer commented 4 years ago
(cfu) ramsesdebeer@cloudshell:~ (ramses-test2)$ libcloudforensics gcp ramses-test2 querylogs --filter='resource.type="gce_instance"'
Traceback (most recent call last):
  File "/home/ramsesdebeer/cfu/bin/libcloudforensics", line 8, in <module>
    sys.exit(Main())
  File "/home/ramsesdebeer/cfu/lib/python3.7/site-packages/examples/libcloudforensics.py", line 143, in Main
    parsed_args.func(parsed_args)
  File "/home/ramsesdebeer/cfu/lib/python3.7/site-packages/examples/gcp_cli.py", line 87, in QueryLogs
    results = logs.ExecuteQuery(args.filter)
  File "/home/ramsesdebeer/cfu/lib/python3.7/site-packages/libcloudforensics/providers/gcp/internal/log.py", line 103, in ExecuteQuery
    gcl_instance_client, 'list', body, throttle=True)
  File "/home/ramsesdebeer/cfu/lib/python3.7/site-packages/libcloudforensics/providers/gcp/internal/common.py", line 219, in ExecuteRequest
    response = request(**kwargs).execute()
  File "/home/ramsesdebeer/cfu/lib/python3.7/site-packages/googleapiclient/discovery.py", line 840, in method
    raise TypeError('Got an unexpected keyword argument "%s"' % name)
TypeError: Got an unexpected keyword argument "resourceNames"
giovannt0 commented 4 years ago

I suspect a change in https://cloud.google.com/logging/docs/reference/v2/rest/v2/entries/list and some new release in https://github.com/googleapis/google-api-python-client

hacktobeer commented 4 years ago

Discovery for this API and method shows it should still be 'resourceNames'... https://www.googleapis.com/discovery/v1/apis/logging/v2/rest?parameters

giovannt0 commented 4 years ago

Right, doesn't make much sense. A quick try changing https://github.com/google/cloud-forensics-utils/blob/2c5b8b50bf19f209fe77309811d466d82302b42c/libcloudforensics/providers/gcp/internal/log.py#L101 to:

body = {
    'body': {
        'resourceNames': 'projects/' + self.project_id,
        'filter': qfilter,
        'orderBy': 'timestamp desc'      
    }
}

Seems to solve the issue. However this breaks the way the pageToken token is currently appended by ExecuteRequest (should then be appended to the dict in 'body'). Doing so might break other API calls that did not change the requests' format. This particular method may need to be handled separately. Will investigate some more!