Open wanghaisheng opened 1 month ago
Hi @wanghaisheng
Sorry, I'm not sure I understood you correctly.
Modin is not responsible for the Ray parameter RAY_memory_usage_threshold
.
Your reproducer seems to be correct. Please contact Ray for more information on this.
I might also suggest that you use Modin cofiguration variable to limit the memory used:
import modin.config as cfg
cfg.Memory.put(2 * 2**30)
or
import os
os.environ["MODIN_MEMORY"] = "2147483648"
Modin version checks
[X] I have checked that this issue has not already been reported.
[X] I have confirmed this bug exists on the latest released version of Modin.
[X] I have confirmed this bug exists on the main branch of Modin. (In order to do this you can follow this guide.)
Reproducible Example
Issue Description
my file is almost 2 G try to set os.environ["RAY_memory_usage_threshold"] =0.9 it says float not support after some filter, tocsv dump give me memory error
Expected Behavior
it should max use 90% of my laptop
Error Logs
Installed Versions