Open eddiebergman opened 3 years ago
For anyone who is looking for the same:
distances, neighbors = des_model._get_region_competence(X_test)
classifiers_predicted_probabilities = des_model._predict_proba_base(X_test)
competences = des_model.estimate_competence_from_proba(query=X_test,
neighbors=neighbors,
probabilities=classifiers_predicted_probabilities,
distances=distances)
selections = des_model.selections(competences)
I haven't looked through all the different models that inherit from BaseDS
but as most (if not all) model types derive from this class, and both _get_region_competence()
and _predict_proba_base
are methods of BaseDS
, the API could be made more user friendly by calling these methods itself, rather than having the user find them. I am happy to help with pull requests if you think it could help the library.
Hello,
Yes, all methods derive from the BaseDS class. I agree with you, the API needs to be improved to be easier to use and allow other users to quickly build new DS techniques by using the basic API methods (_get_region_competence, estimate_competence, etc). My idea is to in the future have those functions exposed and easier to use in a way other users can easily use them to build new techniques. If you can help with that it will be much appreciated. It will definitely help the library and I really believe that having input from users with different needs will help in improving the API.
I also agree that having an example showing how to use those API methods is necessary. But I believe that would be something that we do after refactoring the code.
I'll try have a go at refactoring some of the API calls of the derived models to simplify them and update the documentation, later this week or early next week :)
Great! This will be a good contribution to the library.
If the changes are huge it may be the case to divide the work into multiple smaller pull requests rather than a single one.
Hello,
I'm using
deslib
as part of benchmark paper comparing classifier selection by AutoML tools vs MCS tools for which this library is hugely useful. However, I need to get the competences and selections by both DCS and DES methods indeslib
. There seems to be no examples for what is exposed as public API features inestimate_competence_from_proba(query, neighbors, probabilities, distances=None)
andselection(competences)
.Which of these would be the better way to get the competences and selections:
neighbors
,probabilities
anddistances
. This would help ensure all values are the same as the DCS/DES model would generate but I am unsure which methods to call for these. My gut however says no ifneighbors
,probabilities
have no defaults parameters.sklearn
'sknn
to generateneighbors
anddistances
, manually getting to probabilities of themodels
. This is a lot easier as a non-experienced user ofdeslib
but could lead to inconsistencies with the DCS/DES model.I understand it might be low in the priority but an example or a simplified API would be hugely helpful. If you can point me in the right direction I would be happy to write a pull request with an example or usability improvement!