Route Threshold Optimization Does Not Change Default Threshold Values

aurelio-labs / semantic-router

Superfast AI decision making and intelligent processing of multi-modal data.

https://www.aurelio.ai/semantic-router

MIT License

2.17k stars 224 forks source link

Route Threshold Optimization Does Not Change Default Threshold Values #335

Open vdemchenko3 opened 5 months ago

vdemchenko3 commented 5 months ago

Hi,

What's the best way to interpret unchanged threshold values after optimization? I've ran into a few situations where running route_layer.fit() did not alter the thresholds at all.

Does this mean more data or more iterations are needed or something else entirely?

On a separate note, are there any plans to introduce other optimization methods in the future?

Thanks!

jess-lord commented 3 months ago

also curious about this. Would be nice to see a report on accuracy per class too.

prawiraelang commented 1 month ago

I also noticed this behavior, as discussed in #299. It's been a while since I did the threshold tuning, but seeing many people still talking about this, I suspect it might be a bug in the optimization mechanism.