BCDH / standOffConverter4DARIAH-Campus

This is a work in progress. Once complete, this course will be published on DARIAH-Campus
Creative Commons Zero v1.0 Universal
1 stars 0 forks source link

Standoff Converter: The Missing Link between TEI and NLP

The goal of the course is to help humanities researchers apply NLP (natural language processing) tools and methods on TEI-encoded texts, even though such tools are usually not natively made to work with XML.

Learning outcomes

Upon completion of this course, students will be able to:

TEI and NLP: never the twain shall meet?

TEI and the philological tradition of manual annotation. Digital editions vs. corpora. Yet: digital editions can benefit from NLP annotation: better search and retrieval, indexing, pattern recognition.

But how to do it? Question of scale. We can't do linguistic annotation manually - it would take for ever. But applying NLP tools is not easy because they're usually not made to work natively with XML.

There is a way forward: TEI is flexible.

Inline and standoff annotation

Explain the differences, advantages and disadvantages of storing annotation in the text or separately from it.

Concrete examples.

What is Standoff Converter?

A tool which lets you convert TEI datasets from inline to standoff and vice versa.

Applying NLP tools to TEI-ecoded texts

In this seciton, we'll take you step-by-step through the process of adding lingusitic annotations to TEI-ecnoded texts.

Chose the dataset you want to work with

We provide a sample dataset.

A paragraph or so about the letter(s) we chose.

Set clear annotation goals

What is the end goal? What do we want the final TEI to look like? What attributes and elements are we going to use to annotate lemmas, POS, NER...

Convert the dataset to standoff TEI

Apply NLP tools to standoff TEI

Convert enriched standoff TEI back to inline

Conclusions