search

infn-datacloud / federation-registry-feeder

Python script to populate the Federation-Registry
Apache License 2.0
0 stars 0 forks source link

Avoid script interruption when receiving a ForbiddenException #2

Closed giosava94 closed 6 months ago

giosava94 commented 6 months ago

When the standard use can't access an endpoint of the target provider, due to policy restrictions, the script should not crash. The script should log an error and continue with the other threads.

Currently the script raise an error and stop itself, interrupting all other threads.

Here an example of the error stack printed when having no access to flavor details:

Traceback (most recent call last):
  File "/app/src/main.py", line 44, in <module>
    main()
  File "/app/src/main.py", line 34, in main
    providers = list(filter(lambda x: x, providers))
  File "/usr/local/lib/python3.9/concurrent/futures/_base.py", line 609, in result_iterator
    yield fs.pop().result()
  File "/usr/local/lib/python3.9/concurrent/futures/_base.py", line 439, in result
    return self.__get_result()
  File "/usr/local/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.9/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/app/src/main.py", line 33, in <lambda>
    providers = executor.map(lambda x: get_provider(**x), prov_iss_list)
  File "/app/src/providers/core.py", line 332, in get_provider
    responses = list(filter(lambda x: x, responses))
  File "/usr/local/lib/python3.9/concurrent/futures/_base.py", line 609, in result_iterator
    yield fs.pop().result()
  File "/usr/local/lib/python3.9/concurrent/futures/_base.py", line 439, in result
    return self.__get_result()
  File "/usr/local/lib/python3.9/concurrent/futures/_base.py", line 391, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.9/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/app/src/providers/core.py", line 331, in <lambda>
    responses = executor.map(lambda x: get_idp_project_and_region(**x), inputs)
  File "/app/src/providers/core.py", line 270, in get_idp_project_and_region
    resp = get_data_from_openstack(
  File "/app/src/providers/openstack.py", line 391, in get_data_from_openstack
    compute_service = get_compute_service(
  File "/app/src/providers/openstack.py", line 280, in get_compute_service
    compute_service.flavors = get_flavors(conn)
  File "/app/src/providers/openstack.py", line 99, in get_flavors
    projects = get_flavor_projects(conn, flavor)
  File "/app/src/providers/openstack.py", line 87, in get_flavor_projects
    for i in conn.compute.get_flavor_access(flavor):
  File "/root/.local/lib/python3.9/site-packages/openstack/compute/v2/_proxy.py", line 255, in get_flavor_access
    return flavor.get_access(self)
  File "/root/.local/lib/python3.9/site-packages/openstack/compute/v2/flavor.py", line 154, in get_access
    exceptions.raise_from_response(response)
  File "/root/.local/lib/python3.9/site-packages/openstack/exceptions.py", line 250, in raise_from_response
    raise cls(
openstack.exceptions.ForbiddenException: ForbiddenException: 403: Client Error for url: https://ibisco-endpoint.na.infn.it:42101/v2.1/flavors/318fd069-ffc0-461f-b6ab-6e7e3e3fa8ca/os-flavor-access, Policy doesn't allow os_compute_api:os-flavor-access to be performed.