Note that we should catch all the places where this (acquiring the semaphore unnecessarily) could occur, i.e.: the various multi-file readers, coalesce readers, single-file readers, etc. across all input types (Parquet, ORC, CSV, etc.) Also do we need to worry about this for shuffle, i.e.: is there a case where shuffle read could grab the semaphore for no/empty batches for a task?
This is a lower priority task but it would be good to take care of this so that we don't have odd acquires of the semaphore for something that doesn't need it.
@jlowe had a comment on (https://github.com/NVIDIA/spark-rapids/issues/4392) that was not addressed completely, so I am opening this issue to track:
This is a lower priority task but it would be good to take care of this so that we don't have odd acquires of the semaphore for something that doesn't need it.