medianeuroscience / emfdscore

Fast, flexible extraction of moral information from textual input data.
GNU General Public License v3.0
103 stars 29 forks source link

Why extracted attribute words are so rare using the pat algorithm? #9

Open fishfree opened 3 years ago

fishfree commented 3 years ago

When I execute emfdscore input.csv pat-all-sent.csv pat emfd all sentiment , the input.csv is merged from 350 discourses, but superisingly the output attribute word include only one word "remained"! Are there some bugs?

fhopp commented 3 years ago

The PAT feature is still being developed further, but attributes are detected via spacy SDP engine, and there might not be that many attributes in your input.csv. Try it with the template_input.csv

fishfree commented 3 years ago

@fhopp I tried with the template_input.csv, it still only output one attribute word "remains". I checked the source code. But I'm not familar with NLP, so I don't know how Spacy categorize attribute words? Could you please give some example words?

            if token.text == 'is':
                try:
                    children = list(token.children)
                    word = children[1].lower()
                    if word in emfd.keys():
                        cc_processed[find_ent(str(children[0]),entities)]['attribute_words'].append(word)
                        cc_processed[find_ent(str(children[0]),entities)]['attribute_scores'].append(emfd[word])
                except:
                    pass

            if token.dep_ == 'attr':
                word = token.head.text.lower()
                if word in emfd.keys():
                    for child in token.children:
                        try:
                            cc_processed[find_ent(str(child), entities)]['attribute_words'].append(word)
                            cc_processed[find_ent(str(child), entities)]['attribute_scores'].append(emfd[word])
                        except:
                            pass