Closed rafcis02 closed 2 years ago
@xushiyan : recently we had a fix around aws glue client factory right. is that related to this issue?
@xushiyan : recently we had a fix around aws glue client factory right. is that related to this issue?
no recent fix which resolved the Glue catalog connection issue is not relevant to this.
@rafcis02 can also specify your Glue/Hive configs you're using?
Did you try to run it w/ 0.11.1? I was not able to reproduce your issue with the code-snippet you've pasted.
@alexeykudinkin Environment Description
Running on AWS Glue 2.0
Hudi version : 0.10.1
Spark version : 2.4
Storage (HDFS/S3/GCS..) : S3
The same issue occur using Hudi 0.11.0 but it works fine for 0.11.1. So it seems it has been somehow fixed with the latest release
@xushiyan can you please help close this one?
I got the exact same error and in the end the problem was I missed creating the hive metastore bucket.
Describe the problem you faced
I've been struggling with the failing synchronization with Glue Catalog. I have the process(AWS Glue Job) which reads from Hudi table and then writes to the Huid table as well. Data are being properly saved into S3 bucket but it fails on the Hive table synchronization - it is trying to create database and table that already exist in Glue Catalog. That started happening when we added a code which query Hudi table timeline - without that peace of code it works fine.
To Reproduce You need to have already existing Hudi table on S3 bucket with the table in AWS Glue Catalog Steps to reproduce the behavior:
Here you have the Glue Job script which can use to reproduce it - you need to provide S3 bucket and Glue database name(it cannot be default - you have to create separate database for that). Script contains the part to initialize the table on S3 with sample data. It has a 3 scenarios:
Hive.closeCurrent()
method before upserting dataEach step run as a separate Glue Job Run(including initializing the data sample)
Expected behavior
Hive sync should be successful and job should complete without failure - it should detect that the database and table already exists in Glue catalog and there is no any schema change.
Environment Description
Running on AWS Glue 2.0
Hudi version : 0.10.1
Spark version : 2.4
Storage (HDFS/S3/GCS..) : S3
Additional context
Comparing Scenario 1 and 3 to Scenario 3 I noticed that in Scenario 3 it does not create the AWS Glue Client to sync the table metada. Here is the log that I can find for scenario 1 and 3 but not for 2:
Stacktrace