Talent-Catalog / talentcatalog

https://tctalent.org
GNU Affero General Public License v3.0
10 stars 4 forks source link

New feature: keyword search within specified free text fields #825

Open samschlicht opened 5 months ago

samschlicht commented 5 months ago

I've been logging SQL requests and lately this has been a common reason for them:

The only way to search within certain very useful fields such as company_name in candidate_job_experience, or course_name / institution in candidate_education is elasticsearch.

But with elasticsearch, you obviously get results for hits anywhere in the candidate profile. That creates a lot of chaff.

Two specific examples: Looking for people who have a bachelor degree in Nursing, which was a requirement of the NT pilot in Australia. It's not one of our Education Major dropdown options, so I want to search in candidate_education.course_name. Doing elasticsearch throws up lots of results where 'nursing' shows up in other fields, whereas with SQL I could get straight to the point.

Likewise, someone was after candidates who'd worked for any of the 'big four' accounting firms, to fill a time-sensitive vacancy. Even doing elasticsearch for a term like 'KPMG' throws up 40+ non-pertinent results (people who were featured in a report, once gave a talk, took a training seminar, etc).

camerojo commented 2 months ago

I now think that this simple than it was - when combined with

On the UX side, I think we just add the relevant text fields to the existing filter fields.

I think that this is a hard one for two reasons:

  1. User Experience: It is not clear how we would present the option of searching for text only in certain fields to the user.
  2. Currently we concatenate all relevant text fields from the Postgres DB into a single text field in Elasticsearch - then when searching for text - we just search for text in that single composite text field. That means that it is currently not possible to restrict an Elastic search for text to any particular text field.

So I think we really need to start with a Spike before being able to tackle this - looking into the UX as well as solving the technical challenges.

camerojo commented 2 months ago

This request came from Naji - who is probably currently the only one seeking this functionality - but it does make sense.

Currently, as a work around, we can help Naji with custom SQL on the PostgresDB searching for text in specific fields.

samschlicht commented 1 month ago

Haven't read yet but just posting this article that JC shared -- perhaps this can be done with Postgres text search.