Use iconv library for converting between encoding

[rstemmer]

Currently I use code from the LLVM project. I played a bit with iconv and figured out that this function is exact the function I need :)

Transcode("UTF-8", "UTF-16", inputbuffer, strlen(inputbuffer)+1, outputbuffer, (strlen(inputbuffer)+1) * encodingfactor);

int Transcode(const char *from, const char *to, void *input, size_t inputbytelimit, void *output, size_t outputbytelimit)
{
    iconv_t cd;
    size_t  retval;

    cd = iconv_open(to, from); 
    if(cd == (iconv_t) -1)
    {
        if(errno == EINVAL)
            fprintf(stderr, "\e[1;31miconv_open failed with error \e[1;35mEINVAL\e[1;31m!\e[0m\n");
        else
            fprintf(stderr, "\e[1;31miconv_open failed with unexpected error \e[1;35merrno = %i\e[1;31m!\e[0m\n", errno);
        return -1;
    }

    retval = iconv(cd, (char**)&input, &inputbytelimit, (char**)&output, &outputbytelimit);
    if(retval == (size_t) -1)
    {
        if(errno == E2BIG)
            fprintf(stderr, "\e[1;31miconv failed with error \e[1;35mE2BIG\e[1;31m!\e[0m\n");
        else if(errno == EILSEQ)
            fprintf(stderr, "\e[1;31miconv failed with error \e[1;35mEILSEQ\e[1;31m!\e[0m\n");
        else if(errno == EINVAL)
            fprintf(stderr, "\e[1;31miconv failed with error \e[1;35mEINVAL\e[1;31m!\e[0m\n");
        else
            fprintf(stderr, "\e[1;31miconv failed with unexpected error \e[1;35merrno = %i\e[1;31m!\e[0m\n", errno);
        return -1;
    }

    iconv_close(cd);
    return 0;
}

[rstemmer]

Interface to iconv is implemented. Now I "just" have to remove the old en/decoding and insert the new functions.

[rstemmer]

The new possibilities still needs to be tested and propergated to the command line interface, but with the current feature set and tests it works very well. It also made the code much more clean :)