Ray executor does not gc the input data and the intermediate data of subtask, then the memory usage of worker may be very high. This PR gc the chunk data after every operand finish.
The input data are referenced by Ray core worker in _raylet.pyx, we can't gc them.
Related issue number
Fixes #xxxx
Check code requirements
[ ] tests added / passed (if needed)
[ ] Ensure all linting tests pass, see here for how to run them
What do these changes do?
Ray executor does not gc the input data and the intermediate data of subtask, then the memory usage of worker may be very high. This PR gc the chunk data after every operand finish.
Related issue number
Fixes #xxxx
Check code requirements