A neural parsing pipeline for segmentation, morphological tagging, dependency parsing and lemmatization with pre-trained models for more than 50 languages. Top ranker in the CoNLL-18 Shared Task.
First of all, thanks for your amazing work on Turku NLP!
I have problems reading results from pipeline. The same issues is affecting the provided full_pipeline_server.py of which I was unable to get any responses back to the client. I have located the problem to this line of code: https://github.com/TurkuNLP/Turku-neural-parser-pipeline/blob/master/pipeline.py#L18. output_mod which is appended as the last step seems to drain the queues and not write anything back.
As workaround I have commented that line out from the code but don't consider it as a long time solution. I think full_pipeline_stream.py depends on output_mode being the last step and could be manually appended to the steps there. Writing q_in results back to q_out in output_mode is likely to cause memory leaks if nothing reads the final output queue like in full_pipeline_stream.py.
What do you think about this? I'll create a pull request with the solution described above.
jmnybl
commented
6 years ago
First of all, thanks for your amazing work on Turku NLP!
I have problems reading results from pipeline. The same issues is affecting the provided
full_pipeline_server.pyof which I was unable to get any responses back to the client. I have located the problem to this line of code: https://github.com/TurkuNLP/Turku-neural-parser-pipeline/blob/master/pipeline.py#L18.output_modwhich is appended as the last step seems to drain the queues and not write anything back.As workaround I have commented that line out from the code but don't consider it as a long time solution. I think
full_pipeline_stream.pydepends onoutput_modebeing the last step and could be manually appended to thestepsthere. Writingq_inresults back toq_outinoutput_modeis likely to cause memory leaks if nothing reads the final output queue like infull_pipeline_stream.py.What do you think about this? I'll create a pull request with the solution described above.