Main change: In the actor tagger, the method postag_text is split into two: (1) make_html that generates the html for the GUI, and (2) postag_text_to_table that runs the NLP model does that POS tagging.
Details of the change:
postag_text_to_table creates two dataframes that are indexed by the sentence identifier: self.sentence_df with the sentence text; self.entities_df with the start&end index of the POS tagged entities and the two prominence score metrics
make_html queries the two dataframes created above, filters on selected prominence score and threshold, and creates the html for display.
This is an imperfect solution:
it removes the re-computation for a given selected story, but still re-computes the model and entities whenever a new story is selected.
some functionality is not run anymore, for instance __update_postagging_metrics. -> do we need to rerun this somewhere?
Smaller changes:
use int() for slider and some other things in the OWSNActorAnalysis for compatibility with python>3.9
use testdata from packge for the actor analysis widget
Open tasks
[ ] comments marked with TODO (PR) or TODO NOW
[x] the timing is not better than before: for story 2, changing the selected entities takes 0.3 seconds with the new approach but only 0.15 seconds with the old approach. FIXED: avoid unnecessary steps in the for loop. It is still not faster than than the dictionary approach in the old version, but I think the pandas version will scale better than the dictionary version: https://stackoverflow.com/questions/22084338/pandas-dataframe-performance
[ ] the results are different: for instance, "Mounir" is currently not tagged as a subject in story 2, while it is on the master branch. I think this holds more generally for subject entities, but I need to check.
ThijsVroegh
commented
2 months ago
Issue
Fixes #12
Description of changes
Main change: In the actor tagger, the method
postag_textis split into two: (1)make_htmlthat generates the html for the GUI, and (2)postag_text_to_tablethat runs the NLP model does that POS tagging.Details of the change:
postag_text_to_tablecreates two dataframes that are indexed by the sentence identifier:self.sentence_dfwith the sentence text;self.entities_dfwith the start&end index of the POS tagged entities and the two prominence score metricsmake_htmlqueries the two dataframes created above, filters on selected prominence score and threshold, and creates the html for display.This is an imperfect solution:
__update_postagging_metrics. -> do we need to rerun this somewhere?Smaller changes:
OWSNActorAnalysisfor compatibility with python>3.9Open tasks
TODO (PR)orTODO NOWIncludes