the current parsing loops within all xml tags of TEI verbatim to store in database:
'Speaker.contenttxt' = raw verbatim for each speaker (concatenated if present in different docs)
'Sentence' model, which has:
.contenttxt = raw content of sentence
.contenthtml = styled content, with a/div tags for paraverbal
.i = index of start within text
.o = index of end within text
It's really heavy to compute !
idea: don't store parts of text in the db, rather only use TEI XML file
Solr Indexing
Solr is indexing:
Sentence.contenttxt, to be able to global search for a term, returning exact location (.i / .o)
Speaker.contenttxt, to compute proximity between speakers (used for ngrams, graphs, etc...)
idea: directly index content of TEI documents using solr XSLTPROBLEM ! having to do it all manually, without any help from haystack (update index, ...)
Solr Search
Simple Display
we want to display parts of a document from 'start' to 'end' (using StreamTimelineViz)
fetch array of successive styled sentences
for s in texte.sentence_set.filter(i__range=[start,end]).order_by('i','speaker','n'):
loop in the template
additional styling is made in the django template
spk colors, names
speakers speaking at the same time
idea: XSLT with arguments start/end to directly produce html from TEI XML file
Parsing
the current parsing loops within all xml tags of TEI verbatim to store in database:
It's really heavy to compute !
idea: don't store parts of text in the db, rather only use TEI XML file
Solr Indexing
Solr is indexing:
idea: directly index content of TEI documents using solr XSLT PROBLEM ! having to do it all manually, without any help from haystack (update index, ...)
Solr Search
Simple Display
we want to display parts of a document from 'start' to 'end' (using StreamTimelineViz)
fetch array of successive styled sentences
loop in the template
additional styling is made in the django template
idea: XSLT with arguments start/end to directly produce html from TEI XML file