[BUG] Wrong place to put maxRowsInMemory - Githubissues

crealytics / spark-excel

A Spark plugin for reading and writing Excel files

Apache License 2.0

463 stars 147 forks source link

[BUG] Wrong place to put maxRowsInMemory #863

Open voidlol opened 3 months ago

voidlol commented 3 months ago

Am I using the newest version of the library?

[X] I have made sure that I'm using the latest version of the library.

Is there an existing issue for this?

[X] I have searched the existing issues

Current Behavior

https://github.com/crealytics/spark-excel/blob/main/src/main/scala/com/crealytics/spark/excel/v2/ExcelHelper.scala#L110

Here you are passing maxRowsInMemory option in bufferSize which is wrong :)

Expected Behavior

StreamingReader.builder().rowCacheSize(maxRowsInMemory)

Steps To Reproduce

No response

Environment

- Spark version:
- Spark-Excel version:
- OS:
- Cluster environment

Anything else?

No response