ozborn
commented
6 years ago Can you post a stack trace of this? I'm confused about which client program is creating this error and whether the caller is unhappy getting a blank sentence "document".
I'm also puzzled as to why this error is occurring now in our update and what weird situation yields no document to read AND an empty sentence iterator. Theoretically the caller should be checking hasNext() before calling this... My theory is that it is unhappy returning a blank line and there is some null checking involved upstream, returning null instead of a blank line may be a better bet.
You can also modify it so that there is a parentheses around this.sentences == null || !this.sentences.hasNext() in nextDocument(). Also, return instead of an emtpy string and change the error message.
pzalex
Issue Description
The first line of UimaDocumentIterator nextDocument function (https://github.com/pzalex/deeplearning4j/blob/uimafix/deeplearning4j-nlp-parent/deeplearning4j-nlp-uima/src/main/java/org/deeplearning4j/text/documentiterator/UimaDocumentIterator.java), that checks if resource is available may not have been constructed properly: if (this.sentences == null || !this.sentences.hasNext() && this.reader.hasNext()) { // get the text data; }
the boolean AND that has higher precedence is executed first, so the statement above is equivalent to if [ !this.sentences.hasNext() && this.reader.hasNext()) ] || this.sentences == null (square brackets not for java code). Reads as IF there is no next sentence AND there is a file to read, OR sentences is null, execute the rest. If 'sentences' is null, that would probably generate an exception, trying to get next sentence in the AND clause. Having "sentces == null" in the not parenthesized OR clause, means the following statements trying to retrieve text regardless if file exists or not, what can throw exception.
Statement in hasNext() seem to be correct return this.reader.hasNext() || this.sentences != null && this.sentences.hasNext();
In plain English, return true if there is a document to read, or if there are sentences with a sentence left
At runtime
If hasNext() has not been checked, getNext() can throw exception
Next steps
Get DL4j NLP-UIMA source code into the local clearClinical installation to fix. Commit and push to DL4j uimafix branch if fixed.
Version Information
Contributing
If you'd like to help us fix the issue by contributing some code, but would like guidance or help in doing so, please mention it!