It was reported in #4098 that some of the specified options, like, multiLine, recursiveFileLookup and potentially more, aren't returned as option.multiLine, etc., but instead are expanded into full names, like, spark.sql.dataSourceOptions.multiLine.
This PR changes lookup logic a bit, and if we can't find option.something, then we're looking for all options ending with .something (only if there are no . in the name).
Resolves #4098
Tests
[x] make test run locally
[ ] relevant change in docs/ folder
[ ] covered with integration tests in internal/acceptance
Changes
It was reported in #4098 that some of the specified options, like,
multiLine
,recursiveFileLookup
and potentially more, aren't returned asoption.multiLine
, etc., but instead are expanded into full names, like,spark.sql.dataSourceOptions.multiLine
.This PR changes lookup logic a bit, and if we can't find
option.something
, then we're looking for all options ending with.something
(only if there are no.
in the name).Resolves #4098
Tests
make test
run locallydocs/
folderinternal/acceptance