Open anmyachev opened 1 year ago
Not sure we are able to support all this functionality because Modin treats partitions as immutable objects and we create a new partition in a remote function every time.
Not sure we are able to support all this functionality because Modin treats partitions as immutable objects and we create a new partition in a remote function every time.
For some Modin functions that use several pandas operations in a row in their implementation, this could be useful.
At the moment, none of the options can be changed in Modin processes.
There is a (somewhat peculiar) way, as I said in the pandas 2.0 PR: https://github.com/modin-project/modin/pull/5995#issuecomment-1576166040
At the moment, none of the options can be changed in Modin processes.
There is a (somewhat peculiar) way, as I said in the pandas 2.0 PR: #5995 (comment)
It seems to me that you are talking about a situation where we need to change the value of some option when starting Modin. What if the user needs to change the value after the start? (in the middle of the workload for example)
https://pandas.pydata.org/docs/whatsnew/v2.0.0.html#copy-on-write-improvements
It should be interesting in terms of performance.
The main problem is to enable this option for all processes. At the moment, none of the options can be changed in Modin processes.