When more threads are used than documents exists a NullPointerException occurs.
When calling closeStream(), a NullPointerException appears. This stops the ETL process finishing correctly and results in loss of processed documents.
A workaround is to catch the Exception.
However, as this is a problem can occur everywhere, it would be nicer to somehow control this probably at the ObjectPipeDecoupler.
Also, to say the least, it's rather unexpected to use more threads than documents, it may happen.
Reported at https://github.com/hbz/lobid-resources/issues/1030#issue-502574421:
When more threads are used than documents exists a
NullPointerExceptionoccurs.When calling
closeStream(), aNullPointerExceptionappears. This stops the ETL process finishing correctly and results in loss of processed documents. A workaround is to catch the Exception. However, as this is a problem can occur everywhere, it would be nicer to somehow control this probably at the ObjectPipeDecoupler. Also, to say the least, it's rather unexpected to use more threads than documents, it may happen.