Most long-running tasks in the C++ layer have an optional progress callback argument that can report the progress and potentially terminate the job somewhere in the middle. Python and other language bindings don't currently use that feature, externally slicing the np.ndarray of tasks and updating tqdm indicator after every batch.
That, however, does not apply to cluster, join, save, and load. So we need to pass some functions from Python to C++ bindings, that will be invoked only from one thread to avoid GIL issues.
Can you contribute to the implementation?
[x] I can contribute
Is your feature request specific to a certain interface?
Python bindings
Contact Details
No response
Is there an existing issue for this?
[X] I have searched the existing issues
Code of Conduct
[X] I agree to follow this project's Code of Conduct
Describe what you are looking for
Most long-running tasks in the C++ layer have an optional
progresscallback argument that can report the progress and potentially terminate the job somewhere in the middle. Python and other language bindings don't currently use that feature, externally slicing thenp.ndarrayof tasks and updatingtqdmindicator after every batch.That, however, does not apply to
cluster,join,save, andload. So we need to pass some functions from Python to C++ bindings, that will be invoked only from one thread to avoid GIL issues.Can you contribute to the implementation?
Is your feature request specific to a certain interface?
Python bindings
Contact Details
No response
Is there an existing issue for this?
Code of Conduct