I have a minor bug where I see that the number of rows of a table is doubled in the spark UI. To reproduce, simply create a snowflake table with any number of rows, then, with a spark session do the following:
val dataframe = spark
.read
.format(SNOWFLAKE_SOURCE_NAME)
.options(...)
.load()
dataframe.orderBy(rand()).collect()
If you look at the spark UI in the SQL / Dataframe tab and go into the latest collect row, you should see that the number of rows shown is doubled.
I have a minor bug where I see that the number of rows of a table is doubled in the spark UI. To reproduce, simply create a snowflake table with any number of rows, then, with a spark session do the following:
If you look at the spark UI in the
SQL / Dataframe
tab and go into the latestcollect
row, you should see that the number of rows shown is doubled.