rolkey / indyproject

Automatically exported from code.google.com/p/indyproject
0 stars 0 forks source link

TIdMBCSEncoding GetByteCount() and GetCharCount() broken when using ICONV #228

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago
The GetByteCount() and GetCharCount() method of TIdMBCSEncoding are broken when 
using the ICONV library.

Since iconv() does not support calculating a converted string's length without 
actually converting data, this logic is implemented manually in Indy.  The goal 
was to convert the input data codepoint-by-codepoint, using a small byte buffer 
to save memory without having to allocate a buffer large enough to convert the 
entire input data in one go. So the code calls iconv() in a loop, passing it 
the remaining input data and letting it decode as much as it can into the local 
buffer. However, this logic appears to be flawed because of the following 
condition in the iconv() documentation 
(http://www.gnu.org/savannah-checkouts/gnu/libiconv/documentation/libiconv-1.13/
iconv.3.html):

"
The conversion can stop for four reasons:
...
4. The output buffer has no more room for the next converted character. In this 
case it sets errno to E2BIG and returns (size_t)(-1).
"

If the remaining input data that is passed to iconv() on any given loop 
iteration would exceed the size of the local buffer when converted, I can see 
how this condition would being met.

The loop logic needs to be re-work to address this issue.

Original issue reported on code.google.com by gambit47 on 2 Aug 2012 at 5:48

GoogleCodeExporter commented 8 years ago

Original comment by gambit47 on 4 Feb 2013 at 7:38