Closed proycon closed 4 years ago
As such, this may be an excellent idea. I assume that document with form="explicit" or typegroup=".." attributes are no longer validated correctly by folialint?
I suggest adding an issue about this to libfolia.
I already implemented it in libfolia (the commit references this issue) ;) Still pending release though.
A. I overlooked that. So you took a shortcut by just ignoring the new attributes?
A more solid solution would of course be to add an explicit mode to libfolia/folialint and have those in the output too, when desired. But not a real showstopper.
Yes, the new attributes by definition don't convey any information that needs to be parsed by the libraries, so I can just ignore them. The most important thing is that documents in explicit mode validate properly.
Adding the possibility for explicit serialisation in libfolia is an option yes, but not really a priority or requirement.
Some of the logic in FoLiA is opaque when when looking at the XML structure and is instead specifically handled by the libraries. This concerns for example the handling of default values, most of which are resolved by taking the declaration section into account:
<t>
is current.Making these implicit makes sense as not doing so leads to a great amount of redundancy with even bigger FoLiA files and memory footprint. It does, however, come with a downside: parsers need to implement this logic if they want to fully parse a document.
I propose we implement an alternative "explicit" XML serialisation form (which is just a small superset of the normal form) that makes all these implicit details explicit in the XML to facilitate the job for simpler parsers. It is important to note that both serialisations describe the exact same model and can always be converted from one to the other. The folia validators (foliavalidator, folialint) could perform this conversion if requested.
form="explicit"
in the FoLiA tag. When form is not set to explicit (or absent) altogether, behaviour is unchanged as normal.<t>
will become<t class="current">
)typegroup
attribute that makes explicit what kind of annotation element we are dealing with. Values are: structure, inline, span, higherorder, markup, layer. So<w>
becomes<w typegroup="structure">
,<pos>
becomes<pos typegroup="inline">
. This allows for example xpath expressions like: give me the deepest structural ancestor.IDs are mandatory on all annotation elements (the library will auto-generate IDs where needed, and these IDs will adhere to some kid of convention that marks them as autogenerated (two leading and trailing underscores for example?) so they can be stripped again if needed)(idea discarded for explicit mode because it would add extra information not present in the normal form)One other possibility I'm still on the fence about is explicitly linking from word tokens to their span annotations, using for instance a
<spanref id="entity.1">
in the scope of every<w>
that is part of that entity. This would be the reverse of the<wref>
.The aim of this explicit form is to help parsers, especially those not implementing the full FoLiA logic. The default serialisation remains the 'normal' one. Parsers that can not deal with a document in normal form should themselves invoke foliavalidator/folialint to do the conversion to explicit form prior to parsing it themselves. I'd rather not see this burden shifted to end-users.