datahub-project / datahub

The Metadata Platform for your Data and AI Stack
https://datahubproject.io
Apache License 2.0
9.93k stars 2.94k forks source link

Unable to ingest metada from Hive #11370

Open KewMannDev opened 2 months ago

KewMannDev commented 2 months ago

I have created a datahub instance using the quickstart docker images. When I proceeded to create a Hive source and execute the recipe, I received the following error in the attached log:

log.txt

I tested the Hive connectivity with DBeaver and i was able to connect to hive

To Reproduce Steps to reproduce the behavior:

  1. Go to Ingestion --> Create Source
  2. Click on the Hive option
  3. Enter the necessary details
  4. Click Save and Run

Expected behavior Datahub should be able to ingest metadata from Hive.

Screenshots image

Desktop (please complete the following information):

Additional context command i use to bring up datahub quickstart: python3.8 -m datahub docker quickstart --quickstart-compose-file /opt/datahub/docker-compose.yaml

the custom docker-compose.yaml used: docker-compose.zip

I was able to connect y'day but today after making some changes to the docker-compose.yaml and nuking the old images i received this error. All i did was add the METADATA_SERVICE_AUTH_ENABLED environment variable to the datahub-gms and datahub-frontend images.

Appreciate the help

KewMannDev commented 2 months ago

Just an update, I tried removing the METADATA_SERVICE_AUTH_ENABLED and got the same error

KewMannDev commented 2 months ago

Hi, any updates on this? Thanks

deepgarg-visa commented 1 month ago

After analysing the log file shared, it seems like ingetion module is not able to connect to GMS on localhost:8080 share recipe file also for used for ingestion