Train a named-entity recognition model (via prompt tuning or some other method)

higgood / med-jargon-explain-inator

Forking this so that we can associate tasks with the relevant repo. The ownership of this project belongs to all team members, and not to HIGG. HIGG is only sponsoring to facilitate project management.

2 stars 1 forks source link

Train a named-entity recognition model (via prompt tuning or some other method) #11

Open jonjiang1 opened 6 months ago

jonjiang1 commented 6 months ago

Clarification: We don't need this module to identify new jargon terms, but it needs to:

identify terms/phases with small edit distance from the jargon terms we have, and
determine if a jargon term mention is not relevant given its context.

Note: Reach out to Bridger for help

Steps:

[ ] Identify candidates based on edit distance from the jargon list. This step doesn't require a model.
[ ] For each candidate, determine if it should be explained by our API. This step can be done using prompt tuning or another method.

wammar commented 6 months ago

@XinyiXiang created a PR #17 for this and it was reviewed. Follow up work needed by @XinyiXiang .

wammar commented 6 months ago

Acceptance criteria:

A well-defined interface in the swagger/openapi file for this functionality. The input is text. The output is a list of dictionaries, each of which has the following fields: char_position_in_text, term_length.
An implementation for this interface in FastAPI.

wammar commented 6 months ago

@XinyiXiang : We are blocked on this component so I asked @ecm68 to help deliver this by Thursday May 30.