Support for reading multiple sorted files per bucket

[rahij]

Upstream SPARK-XXXXX ticket and PR link (if not applicable, explain)

Redoing #731 and #738 on the new branch. The upstream PR author has said that it is taking longer since they are planning to automatically detect if a parent operator can take advantage of the sort before creating the bucketed sorted RDD. We will revert this PR when either of these happen:

• The upstream PR is merged
• This PR causes merge conflicts when trying to cherry pick any unrelated upstream change

[rshkv]

For reference, this was the original PR: https://github.com/apache/spark/pull/29625