I am using HWC in SQL mode to read data from Hive Acid table in Spark 2.3. Hive LLAP is not supported in deployed Hive setup hence using SQL mode. I am fetching about 1 GB data with millions of rows. One observation - the Hive query is executed in YARN UI however the Driver / Executor doesn't move to execution of next Spark Action entering a mode of pause. What is cause of it? Is it IO limiting threads between driver and executor for this data fetched?
I am using HWC in SQL mode to read data from Hive Acid table in Spark 2.3. Hive LLAP is not supported in deployed Hive setup hence using SQL mode. I am fetching about 1 GB data with millions of rows. One observation - the Hive query is executed in YARN UI however the Driver / Executor doesn't move to execution of next Spark Action entering a mode of pause. What is cause of it? Is it IO limiting threads between driver and executor for this data fetched?