search

ytsaurus / ytsaurus-spyt

YTsaurus SPYT provides an integration with Apache Spark
Apache License 2.0
9 stars 4 forks source link

Reading with the specified scheme does not work #12

Closed MrSandmanRUS closed 3 weeks ago

MrSandmanRUS commented 1 month ago

Currently, reading with the specified scheme does not work. For example, we have a yt table where one of the columns is of the int type, and we want to immediately read it as a string.

How to reproduce: val sourceDF = spark.read .schema(schema) .yt(cfg.source_yt)

Expected behavior: When reading, you can immediately specify the desired scheme that you want to use.

ykuc commented 4 weeks ago

if The logic is the same for reading yson, It would be great

Alexvsalexvsalex commented 3 weeks ago

The feature was added in the new 2.2.0 version. commit

ykuc commented 3 weeks ago

@Alexvsalexvsalex Will this logic work with yson?