search

knaw-huc / textannoviz

GNU General Public License v3.0
1 stars 1 forks source link

Improve functionality search fragmenter #30

Open svandaalen opened 10 months ago

svandaalen commented 10 months ago

In the current implementation of the fragmenter, the sentence option always stops after a period. This is not always desirable, because this might cause the fragmenter to stop in the middle of the sentence if there is a period after a number, abbreviation, or an initial.

Joris suggests:

We zouden ook kunnen overwegen om maar twee soorten snippets aan te bieden: relatief kort en de hele resolutie, of als medium snippets kiezen voor een vast aantal regels in plaats van het nogal arbitraire criterium van de punt.

svandaalen commented 10 months ago

This sentence functionality was introduced by the experimental ES highlighter plugin. Hayco is in the process of removing this plugin from Broccoli, meaning that we can have a closer look at the default fragmenter functionality in ES.

svandaalen commented 6 months ago

In TAV add fragmenter size of "S", "M", and "L". Make configurable per project in TAV with some arbitrary numbers.