Open alextokarew opened 1 month ago
It is needed to support writing Spark dataframes with StringType columns to YTsaurus tables with Utf8 type columns. Two ways should be supported:
df.write.option("string_to_utf8", "true").yt("//path/to/table")
import yt.type_info as ti; df.write.schema_hint({"some_string_column": ti.Utf8})
It is needed to support writing Spark dataframes with StringType columns to YTsaurus tables with Utf8 type columns. Two ways should be supported:
df.write.option("string_to_utf8", "true").yt("//path/to/table")to convert all string columns to Utf8 type on writing;import yt.type_info as ti; df.write.schema_hint({"some_string_column": ti.Utf8})to convert explicitly specified fields to utf8