Infinite loop when word-joiner is present

[hassanakbar4]

component_Version 2 cli resolution_fixed type_defect | by dev+ietf@seantek.com

---

xml2rfc v2.4.8 goes into an infinite loop when certain illegal characters, specifically &wj; (either word joiner or an unrecognized entity reference), are present.

For example, with the attached XML file, if the following text is present around line 78: ldap:/&wj;/&wj;/&wj;cn=ExampleCA,o=ExampleCo,st=California,c=US

Then, the following output will come out of xml2rfc:

WARNING: Illegal character replaced in string:  
WARNING: Illegal character replaced in string:  
…(continues until process killed)
WARNING: Illegal character replaced in string:  
^CTraceback (most recent call last):
  File "/usr/local/bin/xml2rfc", line 225, in <module>
    main()
  File "/usr/local/bin/xml2rfc", line 210, in main
    pagedwriter.write(filename)
  File "~/Library/Python/2.7/lib/python/site-packages/xml2rfc-2.4.8-py2.7.egg/xml2rfc/writers/base.py", line 1173, in write
    xml2rfc.utils.safeReplaceUnicode(self.r)
  File "~/Library/Python/2.7/lib/python/site-packages/xml2rfc-2.4.8-py2.7.egg/xml2rfc/utils.py", line 301, in safeReplaceUnicode
    element.text = _replace_unicode_characters(element.text)
  File "~/Library/Python/2.7/lib/python/site-packages/xml2rfc-2.4.8-py2.7.egg/xml2rfc/utils.py", line 326, in _replace_unicode_characters
    entity = match.group(1).encode('ascii', 'xmlcharrefreplace')

Therefore, it appears that something is going wrong in _replace_unicode_characters in utils.py.

This infinite loop condition seems to appear both in a locally executed version, and on xml2rfc.ietf.org.

---

Issue migrated from trac:267 at 2021-10-20 18:16:38 +0500

[hassanakbar4]

@{"email"=>"dev+ietf@seantek.com", "name"=>nil, "username"=>nil} uploaded file draft-bad.xml (3.2 KiB)

File with data that causes infinite loop

[hassanakbar4]

@{"email"=>"arusso@amsl.com", "name"=>nil, "username"=>nil} commented

---

Infinite "WARNING: Illegal character replaced in string:" as described above. Seems to be caused by one tab placed in this line in the attached XML file:

192 bits for SHA-256,    256 bits for

where that is

192 bits for SHA-256,<space><tab>256 bits for

using version 2.5.0

[hassanakbar4]

@{"email"=>"arusso@amsl.com", "name"=>nil, "username"=>nil} _uploaded file test_bad_tab.xml (24.6 KiB)_

[hassanakbar4]

@{"email"=>"ietf@augustcellars.com", "name"=>nil, "username"=>nil} changed owner from henrik@levkowetz.com to ietf@augustcellars.com

[hassanakbar4]

@{"email"=>"ietf@augustcellars.com", "name"=>nil, "username"=>nil} changed status from new to closed

[hassanakbar4]

@{"email"=>"ietf@augustcellars.com", "name"=>nil, "username"=>nil} changed resolution from ` tofixed`

[hassanakbar4]

@{"email"=>"ietf@augustcellars.com", "name"=>nil, "username"=>nil} commented

---

We now correctly deal with both the word joiner and the tab in the middle of a CDATA string. In both cases they are modified and we can continue the processing