opendatadiscovery / odd-collectors

Apache License 2.0
8 stars 9 forks source link

Databricks collector fails with AttributeError: can't set attribute #80

Closed paulnim-worldremit closed 2 months ago

paulnim-worldremit commented 2 months ago

The collector appears to run normally, printing results of querying the databricks api then the error below is shown The complete stack is

2024-06-20 20:13:26.976 | DEBUG    | odd_collector.adapters.databricks.client:_get_request:25 - Result of request /api/2.1/unity-catalog/tables is {}
2024-06-20 20:13:27.002 | DEBUG    | odd_collector_sdk.job:log_execution:24 - Traceback (most recent call last):
  File "/app/odd_collector/adapters/databricks/adapter.py", line 45, in get_data_entity_list
    [map_table(self.oddrn_generator, table) for table in schema]
  File "/app/odd_collector/adapters/databricks/adapter.py", line 45, in <listcomp>
    [map_table(self.oddrn_generator, table) for table in schema]
  File "/app/odd_collector/adapters/databricks/mappers/table.py", line 45, in map_table
    table = get_table(table_metadata)
  File "/app/odd_collector/adapters/databricks/mappers/table.py", line 15, in get_table
    table = DatabricksTable(
  File "<string>", line 14, in __init__
AttributeError: can't set attribute

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/odd_collector_sdk/job.py", line 22, in log_execution
    yield
  File "/usr/local/lib/python3.9/site-packages/odd_collector_sdk/job.py", line 93, in start
    async for del_ in self._get_data_entity_list():
  File "/usr/local/lib/python3.9/site-packages/odd_collector_sdk/job.py", line 99, in _get_data_entity_list
    data_entity_lists = await self._adapter.get_data_entity_list()
  File "/app/odd_collector/adapters/databricks/adapter.py", line 60, in get_data_entity_list
    raise MappingDataError(f"Error during mapping: {e}") from e
odd_collector_sdk.errors.MappingDataError: Error during mapping: can't set attribute

2024-06-20 20:13:27.002 | ERROR    | odd_collector_sdk.job:log_execution:25 - [databricks-adapter] failed.
 Error during mapping: can't set attribute