fix(apps/web): Search result content highlighting when token length <= 3 characters

lindylearn / aboutideasnow

Find people to talk to or collaborate with by searching across the /about, /ideas and /now pages of 1000s of personal websites.

https://aboutideasnow.com

MIT License

192 stars 6 forks source link

fix(apps/web): Search result content highlighting when token length <= 3 characters #4

Closed will-holley closed 6 months ago

will-holley commented 6 months ago

Summary

Within search results, ensures that all text matching the query is highlighted.

Changes

40d01efcb13901864c481a74a59d626fa4b5536c: Allows the application to be started without a database connection
a6fe1dabd83b896eaf5854f91552468c20b54674: Prefer the results snippet generated by Typesense; fallback to the present implementation if snippet is not present. It's unclear to me why token.length <= 3 is present on :62 but this is causing the dropped highlights.

Screenshots

Before

After

vercel[bot] commented 6 months ago

@will-holley is attempting to deploy a commit to the LindyLearn Team on Vercel.

A member of the Team first needs to authorize it.

phgn0 commented 6 months ago

Hey @will-holley thank you so much for this fix!

The reason I excluded small highlights previously was because it matched a lot of "I", "at" and similar small words in random places. But using the TypeSense snippet seems to help.

One thing I don't like though is that the snippet always seems to be smaller than the indexed paragraph, so there's context missing :(

At the moment I actually disabled the keyword search in favour of entirely semantic search by changing the query_by key in typesense.ts. I've been alternating between the two approaches a lot, still haven't found good setting so that it considers keywords + meaning. Do you have an idea?

will-holley commented 6 months ago

No problem, @phgn0!

Can you be more specific or provide an example when you say context missing? I'm not sure if I have the same mental picture.

I like exact string matching. For context, the query that originally brought me to the site, "fine art market", is very exact and niche. My hypothesis was that sites containing it belong to people who work in the fine art market; for example, the only exact match belongs to a former engineer at Artsy.net. With the switch to semantic search, I can no longer find this result because results containing "art" rank much higher.

My preference is allowing the user to choose between keyboard and semantic search, for example by using a toggle or by wrapping the query in double quotes to indicate that the user wants an exact keyword match.

will-holley commented 6 months ago

Another consideration: testing locally, I've noticed that semantic matches which don't contain the query won't return any highlights, making it harder to discern why a given result was returned.