search

hortonworks-spark / spark-llap

Apache License 2.0
101 stars 68 forks source link

HiveStreamingDataSource format writes empty string for NULL value in Hive Table #269

Open krvikash opened 5 years ago

krvikash commented 5 years ago

I am trying to write from Kafka Source to Hive Target. I am using "com.hortonworks.spark.sql.hive.llap.streaming.HiveStreamingDataSource" format to write data into Hive table. Table is ORC format and fully transactional. I am using HDP 3.1 cluster.

But What i see that for every NULL value of string data type is written as empty in hive table. Is it the bug or is it the expected behavior?

Attaching Standalone spark program standalone.txt

Create Table Command : create table hive_target (id int, name varchar(255), age int, timestamp1 timestamp);

Table Properties : table_props