googleapis / google-api-python-client

🐍 The official Python client library for Google's discovery based APIs.
https://googleapis.github.io/google-api-python-client/docs/
Apache License 2.0
7.78k stars 2.42k forks source link

Cloud Asset SDK list_next fails for multiple assetTypes filter #2178

Open uripre opened 1 year ago

uripre commented 1 year ago

The Cloud Asset SDK list call with multiple asset types in the assetTypes filter succeeds in the first request, but fails in list_next. See below.

Environment details

Steps to reproduce

  1. Use the Cloud Asset API SDK -> list with multiple asset types
  2. It works for the first request
  3. Try to paginate using list_next
  4. Exception: "ValueError: URL-encoded content contains a repeated value:assetTypes..."

Code example

project_id = 'YOUR_PROJECT_ID'
credentials = ...
asset_types = [
    'firestore.googleapis.com/Database', 
    'storage.googleapis.com/Bucket', 
    'compute.googleapis.com/Disk', 
    'logging.googleapis.com/LogBucket'
] # Just for example

assets_service = discovery.build("cloudasset", "v1", credentials=credentials).assets()

assets_request = assets_service.list(
    parent=f"projects/{project_id}", contentType="RESOURCE", assetTypes=asset_types, pageSize=1 # pageSize is only to ease getting pagination working with less assets; the exception happens without it as well...
)
while assets_request is not None:
    response = assets_request.execute()
    assets_request = assets_service.list_next(assets_request, response)

Stack trace

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[13], line 17
     15 while assets_request is not None:
     16     response = assets_request.execute()
---> 17     assets_request = assets_service.list_next(assets_request, response)

File ~/venvs/project/lib/python3.11/site-packages/googleapiclient/discovery.py:1322, in createNextMethod.<locals>.methodNext(self, previous_request, previous_response)
   1318 request = copy.copy(previous_request)
   1320 if isPageTokenParameter:
   1321     # Replace pageToken value in URI
-> 1322     request.uri = _add_query_parameter(
   1323         request.uri, pageTokenName, nextPageToken
   1324     )
   1325     logger.debug("Next page request URL: %s %s" % (methodName, request.uri))
   1326 else:
   1327     # Replace pageToken value in request body

File ~/venvs/project/lib/python3.11/site-packages/googleapiclient/_helpers.py:207, in _add_query_parameter(url, name, value)
    205     return url
    206 else:
--> 207     return update_query_params(url, {name: value})

File ~/venvs/project/lib/python3.11/site-packages/googleapiclient/_helpers.py:184, in update_query_params(uri, params)
    167 """Updates a URI with new query parameters.
    168 
    169 If a given key from ``params`` is repeated in the ``uri``, then
   (...)
    181     The same URI but with the new query parameters added.
    182 """
    183 parts = urllib.parse.urlparse(uri)
--> 184 query_params = parse_unique_urlencoded(parts.query)
    185 query_params.update(params)
    186 new_query = urllib.parse.urlencode(query_params)

File ~/venvs/project/lib/python3.11/site-packages/googleapiclient/_helpers.py:161, in parse_unique_urlencoded(content)
    156     if len(value) != 1:
    157         msg = "URL-encoded content contains a repeated value:" "%s -> %s" % (
    158             key,
    159             ", ".join(value),
    160         )
--> 161         raise ValueError(msg)
    162     params[key] = value[0]
    163 return params

ValueError: URL-encoded content contains a repeated value:assetTypes -> firestore.googleapis.com/Database, storage.googleapis.com/Bucket, compute.googleapis.com/Disk, logging.googleapis.com/LogBucket

Thanks!

inigo-a-martinez-de-ubago-db commented 8 months ago

Any comment about this? I'm facing the same issue, but don't know how to solve it. Thanks!