Only suggest topics associated with at least one board report

[hancush]

Regardless of whether the query is the preferred or alternative term, we should only suggest topics if they're associated with board reports.

[hancush]

Update: I tried to accomplish this using the minDocs filter in our API call, which I took to be an indication of how many documents were tagged with a term in SmartLogic. Per Steve, that's not quite right:

Mindocs is a filter where we could indicate how many resources in a system like Legistar had that concept attached. This value had to be periodically calculated from the source system and set for each concept. For Metro they’re all zero so not helpful.

He suggested that we might perform and store this count on our end, but in my view, it makes a lot more sense for it to be performed at the SmartLogic level, since their system is already set up to ingest this data.

@shrayshray, Steve mentioned that he'd need to extend the SmartLogic-Legistar bridge for this count to be accurate, and that he'd talk to you all about it. Once it's in place, this will be a trivial change on our end, but until then, I'd unfortunately consider us blocked.

[shrayshray]

@hancush would it be possible to do something like query the Legistar API each night to get a list of all terms used in MatterIndexes (exclude duplicates) and use that list to check and exclude those which are not used?

[hancush]

@shrayshray It would be possible for us to do that, however it would be a larger change. If the SmartLogic-Legistar bridge is already set up to store that information and already does a nightly sync with Legistar, it would make more sense for that operation to happen at the SmartLogic level. Then, we could make a tiny change on our end to exclude unused indexes from suggestions.

With all that said, I think it would good idea to have a call with you and Steve to discuss options and figure out what makes most sense and will be most cost effective for you all.

[hancush]

Look into adding indication of how frequently a suggested term is used (or even ordering suggestions by frequency).

Goal: Minimize dead end suggestions, without preventing legitimate searching.

[hancush]

@shrayshray @camachoo I've staged this improvement. You can compare suggestions (from the search box) and related terms (on the search results page) between the staging site and the live site.

Here's an example using my go-to term, "red".

Original (live site):

Improved (staging):

What do you think? Does this represent an improvement?

[hancush]

@shrayshray Do you think you'll have time to review this before next week's meetings? If not, I'll go ahead and deploy the update to the meeting status logic so we can test out the revision to the concluded tag. (Can't have both at once.)

[shrayshray]

@hancush i need to take a closer look at this, so please deploy the update to the meeting status logic instead.

[hancush]

Will do, @shrayshray, thank you!

[shrayshray]

@hancush could you help me understand why limiting the suggestions in the search box to only terms tagged on at least one board report would impact what shows in the related terms section of the search results? I'm extra confused about why this limitation would lead to more related terms listed, rather than fewer. @camachoo Do you think it would be better to include terms which aren't tagged on any reports in the search suggestions (E.g., all the stages of the Red Line, so people know other stages exists, even if they return zero results)? The example Hannah used in the screen shots has me second guessing myself.

[hancush]

@shrayshray Oh, I didn't realize you'd like related terms to stay the same! That's not a problem – I can preserve the original behavior of related terms while restricting search suggestions to only terms related to at least one board report. How's that sound?

[shrayshray]

@hancush yes, let's stick with what we had before for the related terms section. @camachoo could you weigh in on my previous comment on the suggestions in the screenshot above?

@camachoo Do you think it would be better to include terms which aren't tagged on any reports in the search suggestions (E.g., all the stages of the Red Line, so people know other stages exists, even if they return zero results)? The example Hannah used in the screen shots has me second guessing myself.

[hancush]

Table for now. We'll see how federated search tech works via testing, then circle back.