Menelau
commented
1 year ago Hello @SachiniEMSPE sorry for the late response.
If you want to get which classifiers were selected for each test example, you will need to do a small modification in the code in order to save this information, unfortunately, we never returned this info with the predict method to follow up scikit-learn standards of just returning the predictions and nothing else. I will see if I can either allow returning this info after calling predict by adding an argument to predict requesting to return this info.
The flow of all DS techniques in the library is that it will compute the region of competence once (this is done in the BaseDS class at the beginning, which can be found in the "base.py" method) and this info will be passed down to either a DCS or DES technique for selecting and combining single or multiple models. Inside the DES and DCS folder, you can find the base classes for DCS and DES, which implement the "classify_with_ds" method. This method controls the basic flow of DS techniques for estimating the competence, selecting, and combining models. What you need to do in terms of coding to have access to such models is to, check at the line of code that is calling the "select" method and returning the array "selected_classifiers". This is a (n_samples, n_classifiers) binary array containing True whether a classifier is selected for a given sample and False otherwise. If you store this array or modify the code to return this variable, you will have access to this info after calling the predict method. You could do the same in the case you want also to store/analyze the competence measure estimated by each classifier. After calling the "estimate_competence" method in the same base classes, it returns an array with shape (n_samples, n_classifiers) with each position corresponding to the estimated competence level for a single base model. So saving/returning this info you can also obtain this info if you need to analyze it later.
SachiniEMSPE
Hi! I have a question on DCS methods. It would be great if you could help me with this. Thank you! Q: Consider a test instance and a DCS method (e.g., OLA). Is there any function/method available yet to find the classifier that gives the decision at each test instance?