Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.
[x] Lint and unit tests pass locally with my changes
[x] I have added tests that prove my fix is effective or that my feature works
[x] I have added necessary documentation (if appropriate)
Further comments
I cannot see dramatic changes in performance (spark 3.3, spark 3.2), but I can guess that in the future versions of spark more and more things will be disabled/simplified when it is turned on. On spark 3.4/3.5 performance improvement should be significant.
Proposed changes
As described in #404
Checklist
Put an
xin the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.Further comments
I cannot see dramatic changes in performance (spark 3.3, spark 3.2), but I can guess that in the future versions of spark more and more things will be disabled/simplified when it is turned on. On spark 3.4/3.5 performance improvement should be significant.