open-metadata / OpenMetadata

OpenMetadata is a unified metadata platform for data discovery, data observability, and data governance powered by a central metadata repository, in-depth column level lineage, and seamless team collaboration.
https://open-metadata.org
Apache License 2.0
5.26k stars 994 forks source link

Unable to ingest metadata for PinotDB #17963

Open dzhou26 opened 4 days ago

dzhou26 commented 4 days ago

Affected module This is impacting Ingestion Framework

Describe the bug Not able to ingest metadata for Pinot DB. The ingestion task failed with the following error:

File "/home/airflow/.local/lib/python3.10/site-packages/metadata/ingestion/source/database/common_db_source.py", line 210, in yield_database
    name=EntityName(database_name),
File "/home/airflow/.local/lib/python3.10/site-packages/pydantic/root_model.py", line 71, in __init__
    self.__pydantic_validator__.validate_python(root, self_instance=self)
pydantic_core._pydantic_core.ValidationError: 1 validation error for EntityName
    Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]

If a database name is specified, the ingestion will not pass the test connection step:

The above exception was the direct cause of the following exception:
Traceback(most recent call last):
    File "/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/engine/base.py", line 1910, in _execute_context
        self.dialect.do_execute(
    File "/home/airflow/.local/lib/python3.10/site-packages/sqlalchemy/engine/default.py", line 735, in do_execute
        cursor.execute(statement, parameters)
    File "/home/airflow/.local/lib/python3.10/site-packages/pinotdb/db.py", line 57, in g
        return f(self, *args, **kwargs)
    File "/home/airflow/.local/lib/python3.10/site-packages/pinotdb/db.py", line 513, in execute
        return self.normalize_query_response(query, r)
    File "/home/airflow/.local/lib/python3.10/site-packages/pinotdb/db.py", line 57, in normalize_query_response
        raise exceptions.DatabaseError(
pinotdb.exceptions.DatabaseError: Error when querying {"sql": '/* {"app": "OpenMetadata", "version": "1.5.3.1"} */\nSELECT 42'} from [https://localhost:443/default/query/sql](https://localhost/default/query/sql)

To Reproduce Set up a Pinot DB connection by filling out the fields in UI. Username: username Password: password Pinot Broker Host And Port: localhost:443 Pinot Controller Host And Port: https://localhost:443 PinotDBConnection Advanced Config -> Connection scheme: pinot+https

Expected behavior Successfully pull metadata from Pinot DB

Version:

Additional context N/A