Recognizing if a sentence needs coreference

[Gautam-Rajeev]

One of the parts of setting up neural coreference is so check if a sentence needs coreference.
The following code does that by recognizing determiners and pronouns.

We need to integrate this with the coreference layer and also be able to recognize sentences that need coreference even without containing below words as determiners

!pip install nltk

import nltk
nltk.download('punkt')
nltk.download('averaged_perceptron_tagger')

def has_determiners_or_pronouns(sentence):
    tokens = nltk.word_tokenize(sentence)
    tagged_tokens = nltk.pos_tag(tokens)

    determiners = ["this", "that", "these", "those", "my", "your", "his", "her", "its", "our", "their"]
    pronouns = ["he", "she", "it", "they", "we", "you", "i", "me", "him", "her", "us", "them"]

    for token, tag in tagged_tokens:
        if tag.startswith("DT") and token.lower() not in ["a", "an", "the"]:
            return True  # Sentence has determiners

        if tag.startswith("PRP") and token.lower() not in ["he", "she", "it"]:
            return True  # Sentence has pronouns

    return False  # No determiners or pronouns (excluding "a", "an", "the")

# Example usage:
sentence = "This is my cat. She is very playful."
 has_determiners_or_pronouns(sentence)

[VaibhaveS]

Hi, instead of detecting whether a sentence needs coreference or not, can we run coreference by default but apply it only if the confidence interval is say greater than a threshold like 75%?

[Gautam-Rajeev]

@VaibhaveS, yes. Ideally would like the coreference module to recognize and give no coreference (same sentence) when no coreference is required. This is a temporary solution to not hit GPT for all queries as we are currently using GPT for neural coreference in AmaKrushAI.

[ShubhamBhut]

Hey @GautamR-Samagra I am interested in working on this issue. I think we can use neuralcoref other 3rd party modules for this. They appears to work fine. Currently I am not seeing anything for co-reference layer in the codebase. Can you pls elaborate more on how coreference module should be set up ?

[Gautam-Rajeev]

Hey @GautamR-Samagra I am interested in working on this issue. I think we can use neuralcoref other 3rd party modules for this. They appears to work fine. Currently I am not seeing anything for co-reference layer in the codebase. Can you pls elaborate more on how coreference module should be set up ?

This ticket is now merged with https://github.com/Samagra-Development/ai-tools/issues/42 @ShubhamBhut you can check that out and follow the progress there