Begins an implementation of Layer to wrap the List[Entities] and allow for more intuitive slicing (e.g. doc.sentences[:3].text instead of [sent.text for sent in doc.sentences]. This addresses #24 .
The new data structure is implemented in papermage/magelib/layer.py and inherits from python's UserList. The changes into integrate Layer were mainly made in the Document data structure.
One design decision to consider is what to do with chained access: e.g. doc.pages.paragraphs.sentences.tokens. Currently, each access creates a new layer, so doing the above would create a four-dimensional list. Two consequences of this decision:
To get the first token, you would have to write doc.pages.paragraphs.sentences.tokens[0][0][0][0], which is a bit ugly.
doc.pages.paragraphs.pages does not return the original Layer of pages., which is a bit uninutitive
The main question is: "Should chained accessing return the union of all of the entities in a single layer or should it return the entities in the shape of the chained accessing?"
As another example, if the doc is
Paragraph 1: "I am. I was."
Paragraph 2: "You are. You were."
Sentence 1: "I am."
Sentence 2: "I was."
Sentence 3: "You are."
Sentence 4: "You were."
Which should doc.paragraphs.sentences.text return?
Begins an implementation of
Layerto wrap theList[Entities]and allow for more intuitive slicing (e.g.doc.sentences[:3].textinstead of[sent.text for sent in doc.sentences]. This addresses #24 .The new data structure is implemented in
papermage/magelib/layer.pyand inherits from python'sUserList. The changes into integrateLayerwere mainly made in theDocumentdata structure.One design decision to consider is what to do with chained access: e.g.
doc.pages.paragraphs.sentences.tokens. Currently, each access creates a new layer, so doing the above would create a four-dimensionallist. Two consequences of this decision:doc.pages.paragraphs.sentences.tokens[0][0][0][0], which is a bit ugly.doc.pages.paragraphs.pagesdoes not return the originalLayerof pages., which is a bit uninutitiveThe main question is: "Should chained accessing return the union of all of the entities in a single layer or should it return the entities in the shape of the chained accessing?"
As another example, if the doc is
Which should
doc.paragraphs.sentences.textreturn?