Basic ranking methods for results organization module

nasaharvest / dora

Domain-agnostic Outlier Ranking Algorithms (DORA) - SMD cross-divisional use case demonstration of AI/ML

MIT License

12 stars 3 forks source link

Basic ranking methods for results organization module #18

Open hannah-rae opened 3 years ago

hannah-rae commented 3 years ago

We discussed having some basic methods for returning a list of items with their outlier scores, which should be implemented as separate results organization modules:

[ ] Output list of items in the order they are returned by the algorithm (algorithm-order)
[ ] Output list of items in the order they are read by the data loader and passed to the algorithms (item- or original-order)
[ ] Output list of items in order of ascending outlier score (ascending-order)
[x] Output list of items in order of descending outlier score (descending-order)

hannah-rae commented 3 years ago

We talked early on about having different options for what order to save the results in. Currently the save_scores module saves samples in descending outlier order (most outlier-y to least). We could add a parameter that would allow the user to specify the order that the samples are written in the csv based on these four options. I'm curious to hear from others if you agree with that solution and if you think it would be useful still?

cc @stevenlujpl @wkiri @urebbapr @bdubayah

wkiri commented 3 years ago

@hannah-rae In terms of saving time/effort, it seems like the user can already achieve all of these by sorting on different columns in the output .csv file, right? Also I believe the option currently implemented is the first one (algorithm-order) rather than sorting by score.

stevenlujpl commented 3 years ago

@hannah-rae I am not sure if I understand the algorithm-order and item-order methods. in terms of supporting ascending-order, we may also need to modify the outlier ranking algorithms.