Closed foolo closed 5 years ago
Suggested change has been implemented.
Overall performance of the XML library is now slightly worse (in most cases, XML elements contain merged text and now there is an extra cost in creating a StringBuilder) but for merging skeletons and converting to XLIFF 2.0 the change may be relevant.
Nice! Now the calls take 15, 14, and 16 seconds instead of the previous 13, 13, and 77, so the result is as predicted. BTW, I needed to change the module name in convert.sh (it was still using the old xliffFilters name).
I noticed that the -2.0 option in combination with -embed (which is my use case) makes the Convert step takes a very long time. Maybe you are already aware of it, but anyway here comes some measurements and a small investigation. Some examples (all done with the same test.docx, (4.3 MB, 4400 words)):
without -embed flag: ./convert.sh -file test.docx -srcLang da -tgtLang sv -2.0 13 seconds
without -2.0 flag: ./convert.sh -file test.docx -srcLang da -tgtLang sv -embed 13 seconds
with both -2.0 and -embed flag: ./convert.sh -file test.docx -srcLang da -tgtLang sv -2.0 -embed 77 seconds
I did some debugging and it is one particular call to com.maxprograms.xml.Element.mergeText() that takes about 1 minute to complete, and it seems like the bottleneck is this line: https://github.com/rmraya/OpenXLIFF/blob/master/src/com/maxprograms/xml/Element.java#L167
When the mergeText() is run for the
<internal-file>
element (i.e all the base64 skeleton data), thecontent
member is a big vector (37000 lines in my case) which is then concatenated line by line to a new string: t.setText(t.getText() + ((TextNode) n).getText());It can probably be improved fairly easily, so that it runs almost instantly, for example by using a StringBuilder for concatenation.