It seems that Tyra always rank by avg, it doesn't care about the metric

[nanounanue]

The culprit lines:

https://github.com/dssg/tyra/blob/8d6dce32cc0557a8bc91cfa95634eff7f089dbac/webapp/query.py#L59-L64

[nanounanue]

A possible solution for cache_table after this gets fixed is:

with models as (
     SELECT
         model_group_id,
         m.model_comment,
         metric || parameter as metric_parameter,
         avg(e.value) as average_performace
         --e.value,
         --e.evaluation_start_time ::DATE,
         --m.run_time,
     FROM results.models AS m
     JOIN results.evaluations AS e USING (model_id)
     -- Sometimes you want to "predict forward", we don't know if Triage supports this
     WHERE evaluation_start_time = train_end_time :: TIMESTAMP
     GROUP BY model_group_id, m.model_comment, metric_parameter
),

ranked_models as (
     select *,
     row_number() over w as rank
     from models
     window w as ( partition by metric_parameter order by average_performace desc )
)

select * from ranked_models where rank < 20;

Via @k1aus

[tweddielin]

@nanounanue @k1aus would you like to create a new branch and send the PR?