search

GoogleCloudDataproc / hive-bigquery-storage-handler

Hive Storage Handler for interoperability between BigQuery and Apache Hive
Apache License 2.0
19 stars 11 forks source link

INSERT with no result when using Tez #2

Open roderickyao opened 5 years ago

roderickyao commented 5 years ago

When using Tez as Hive execution engine, INSERT statement will return successful but no rows are inserted. This works under MapReduce engine.

Tez somehow does not seem to call OutputCommitter at all.

alfonsonishikawa commented 4 years ago

Happens the same to me. Any fix/workaround? When I execute it from Hive View (HDP) works, but from Oozie does not work. With execution engine mr it works. The execution is not shown in Tez View neither when launching with Oozie.

vishal2535 commented 2 years ago

I am facing same issue with execution engine MR. It's returning success but no rows are inserted. Any idea? Insert into bq_test values ('Vishal', 33); WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases. Query ID = hduser_20220214125524_7b6db0f8-9792-4acb-b4f6-6e4de208f2fa Total jobs = 1 Launching Job 1 out of 1 Number of reduce tasks is set to 0 since there's no reduce operator Job running in-process (local Hadoop) 2022-02-14 12:55:27,341 Stage-3 map = 100%, reduce = 0% Ended Job = job_local758927901_0001 MapReduce Jobs Launched: Stage-Stage-3: HDFS Read: 7 HDFS Write: 7 SUCCESS Total MapReduce CPU Time Spent: 0 msec OK Time taken: 3.203 seconds

michTalebzadeh commented 2 years ago

I believe with mr as engine, there is the issue of OutputCommitter not being called somehow, something probably to do with jar.

vishal2535 commented 2 years ago

@michTalebzadeh Yes, I noticed the same behavior. Let me know if you find any solution for this.