There's a deprecation warning related to the parallelism argument in Ray 2.10. The warning suggests using override_num_blocks instead. This issue occurs in read_api.py at line 3087.
Steps to Reproduce:
Set up an environment with Ray 2.10.
Use the awswrangler.s3.read_parquet function with ray_args that includes the parallelism parameter.
Observe the warning message.
Code Example:
import awswrangler as wr
# Example usage of read_parquet with ray_args including parallelism
df = wr.s3.read_parquet(
path="s3://bucket/path/",
ray_args={"parallelism": 10}
)
Expected Behavior:
The read_parquet function should accept the appropriate parameter for specifying parallelism without generating a deprecation warning.
Suggested Fix:
Update the read_parquet function to use override_num_blocks instead of parallelism when passing ray_args.
There's a deprecation warning related to the parallelism argument in Ray 2.10. The warning suggests using override_num_blocks instead. This issue occurs in read_api.py at line 3087.
Steps to Reproduce:
awswrangler.s3.read_parquet
function withray_args
that includes theparallelism
parameter.Code Example:
Expected Behavior: The read_parquet function should accept the appropriate parameter for specifying parallelism without generating a deprecation warning.
Suggested Fix: Update the read_parquet function to use override_num_blocks instead of parallelism when passing ray_args.