Closed derekahuang closed 1 year ago
@derekahuang hello,
Yes, we are interested in having such function vectorized. In fact, it is one of the issues currently in the repository (issue #71 )
It will be very appreciated if you can send a PR to add this functionality to the library. The methods based on diversity are much slower than the other DS techniques due to this loop not being vectorized yet. If some of its functionality are not trivially vectorized, another solution would be .o write this part of code as a Cython implementation
Are you OK with this interface, since we are only batching one classifier? Would I create a new file, like diversity_batched.py
?
@Menelau i've opened a PR: https://github.com/scikit-learn-contrib/DESlib/pull/273
@derekahuang great, I will check it asap and get back with comments and/or merge.
Thanks for contributing with this feature!
Fixed with PR #273
Hi there,
I am using DESlib for large scale diversity calculations and I'm running into perf issues with the for loop in
_process_predictions
. I was wondering if we have thought about vectorizing this function for bulk calculations. I have reimplemented a vectorized version of the diversity functions that accepts the parametersbut it is less general as
y_pred1
andy_pred2
are no longer swappable. This returns an array of shape(n_classifiers,)
that contains the calculations ofy_pred1
with every row ofy_pred2
. If the maintainers are interested in this let me know, I am more than happy to help integrate this into the library!