I'm wondering the purpose of defining data type in catalog. Looks like if the data type in catalog doesn't match the data type of the corresponding dataframe's column, then the data type of the dataframe's column is applied when writing to HBase. For example, a dataframe has an integer column named 'col1', and the type of this column in catalog is String, then the integer will be put into HBase. Shc doesn't do data type conversion for you from my understanding. If so, what's the point of defining the data type in catalog?
I'm wondering the purpose of defining data type in catalog. Looks like if the data type in catalog doesn't match the data type of the corresponding dataframe's column, then the data type of the dataframe's column is applied when writing to HBase. For example, a dataframe has an integer column named 'col1', and the type of this column in catalog is String, then the integer will be put into HBase. Shc doesn't do data type conversion for you from my understanding. If so, what's the point of defining the data type in catalog?