search

mars-project / mars

Mars is a tensor-based unified framework for large-scale data computation which scales numpy, pandas, scikit-learn and Python functions.
https://mars-project.readthedocs.io
Apache License 2.0
2.68k stars 325 forks source link

Fix duplicate execution #3301

Closed fyrestone closed 1 year ago

fyrestone commented 1 year ago

What do these changes do?

Some chunks are copied when tiling (e.g. copied chunks with new chunk index), these copied chunks can't be treated as executed, so they will be regenerated. This PR use chunk key instead of chunk object to check whether it is executed.

Related issue number

Fixes https://github.com/mars-project/mars/issues/3299

Check code requirements