test_from_utf16: Assertion `res_len == strlen(out)' failed

[hotice]

Using the latest Freshplayerplugin Git code, some tests fail (tested in Ubuntu 14.04, 15.04 and 15.10; it does not occur on Ubuntu 16.04).

Edit: this also occurs with version 0.3.4 (I didn't test older versions - until now tests were disabled in my PPA because it was failing back when I created the package due to non-freshplayerplugin related issues)

From the build log:

Test project /«BUILDDIR»/freshplayerplugin-0.3.4+git20160228/build/tests
    Start 1: test_header_parser
1/9 Test #1: test_header_parser ...............   Passed    0.06 sec
    Start 2: test_ppb_char_set
2/9 Test #2: test_ppb_char_set ................***Exception: Other 15.12 sec
default charset = UTF-8
to utf16: all ASCII
to utf16: basic UTF-8
to utf16: wrong UTF-8 with error
from utf16: all ASCII
to utf16: non-ASCII all correct
to utf16: non-ASCII PP_CHARSET_CONVERSIONERROR_FAIL
to utf16: non-ASCII PP_CHARSET_CONVERSIONERROR_SKIP
to utf16: non-ASCII PP_CHARSET_CONVERSIONERROR_SUBSTITUTE
test_ppb_char_set: /«BUILDDIR»/freshplayerplugin-0.3.4+git20160228/tests/test_ppb_char_set.c:165: test_from_utf16: Assertion `res_len == strlen(out)' failed.

    Start 3: test_ppb_flash_file
3/9 Test #3: test_ppb_flash_file ..............   Passed    0.03 sec
    Start 4: test_ppb_url_request_info
4/9 Test #4: test_ppb_url_request_info ........   Passed    0.03 sec
    Start 5: test_ppb_x509_certificate
5/9 Test #5: test_ppb_x509_certificate ........   Passed    0.02 sec
    Start 6: test_uri_parser
6/9 Test #6: test_uri_parser ..................   Passed    0.03 sec
    Start 7: test_ppb_net_address
7/9 Test #7: test_ppb_net_address .............   Passed    0.02 sec
    Start 8: test_config_parser
8/9 Test #8: test_config_parser ...............   Passed    0.03 sec
    Start 9: test_thread_specifier
Errors while running CTest
9/9 Test #9: test_thread_specifier ............   Passed    0.03 sec

89% tests passed, 1 tests failed out of 9

Total Test time (real) =  15.39 sec

The following tests FAILED:
      2 - test_ppb_char_set (OTHER_FAULT)
tests/CMakeFiles/check.dir/build.make:52: recipe for target 'tests/CMakeFiles/check' failed

Complete build log for Ubuntu 15.10: https://launchpadlibrarian.net/253277906/buildlog_ubuntu-wily-amd64.freshplayerplugin_0.3.4+git20160228-1~webupd8~wily~0_BUILDING.txt.gz

[i-rinat]

Is there any way to reproduce build environment from those build machines?

I tried to run tests on own machine (Debian testing), in Docker image of Ubuntu 16.04, in full installation of Ubuntu 15.04 in virtual machine. All without test failures.

That exact test uses iconv libraries, but those are installed with libc6, and I don't know any sane way to have them deleted. Also test took 15 seconds, which is extremely long. I'm missing something; still don't know what.

[hotice]

My initial bug report was updated shortly after submitting it. It doesn't fail on Ubuntu 16.04 but on Ubuntu 15.10, 15.04 and 14.04. Ubuntu 16.04 is the only one in which the test doesn't fail.

As for reproducing the exact build environment, I'm not sure what you could do other than creating a Launchpad PPA.

[ItachiSan]

Using apt-get download the package ubuntu-dev-tools and then create a builder using pbuilder-dist trusty create To automagically fetch the sources, you can use my PPA here: https://code.launchpad.net/~itachi-san/+archive/ubuntu/freshplayerplugin If you add it, you can fetch the sources with apt-get source freshplayerplugin (Else, I can provide a better way to do it.) After that, you will have a .dsc file. To try building it, you just have to do pbuilder-dist trusty build NAME_OF_DSC_FILE.dsc (It should find the dsc file with tab completion anyways)

You can contact me for any issues, here, on mail or else (Telegram maybe)

[i-rinat]

I managed to create a PPA a while ago. With a number of trials it was found that in failing test iconv replaces unknown symbols not with expected question sign '?', but with '\0', which caused strlen() to report a wrong string length. I also saw similar bug in a Docker environment. I guess something with locales support causing it.

Instead of trying to workaround that case, I decided to move from iconv to icu library. It should work then the same in any locale. Also Chromium uses icu, and therefore PPB_CharSet supports same encoding names as icu does.

[ItachiSan]

Nice! Hope to have working builds soon ;)

[i-rinat]

Changed charset conversion to use ICU (libicu-dev in Debian) in 771bae0d6594c103e359a08628c4dfa75a576333.

I didn't try to upload to PPA yet to test if it works now. But I expect it to pass tests now.

[i-rinat]

Oops. Looks like "to UTF-16" part is now broken. Reverted that commit.

[ItachiSan]

I will disable the test for now for my PPA builds! Apart that, I will check this issue to notice when conversion to ICU is done :)

[i-rinat]

Conversion to UTF-16 wasn't broken. It was "gb2312-80" encoding that was broken. Or not broken. Depends on point of view.

There is an encoding with name "gb2312-80". It has only hieroglyphs in it, there is no even latin letters. But from what I see in dumps, some swf try to convert a text that looks like ASCII text from gb2312-80 to UTF-16, which ICU fails to do — it's not possible. That encoding in its true form is so rare that iconv doesn't know about its existence, but ICU does.

It turned out, that today name "gb2312-80" is used as an alias for gb2312, which in fact, a different encoding. Heh, Chinese Internet.

Added an encoding alias conversion routine in 29915d3bb73cf63acb4f93dfd7c6b05802ea56fd and 4776eeb1a1cf6fac81fa7c3b46e4a6ac8829953c, it should be fine now with gb2312-80 (tested) and other encodings (not tested).

[ItachiSan]

@i-rinat my PPA builds works fine. I only have issues on precise, because 'libicu-dev' doesn't have the needed pkg-config files. I uploaded to the PPA a fixed version of ICU which includes pkg-config data. Other versions work fine. Everything should work now :+1: P.S. should I send a pull request with my PPA added in docs, for people who want latest builds?

[i-rinat]

I've finally tried to upload new version to a PPA to test, and it builds fine on Xenial, while versions before fixes were failing.

[i-rinat]

should I send a pull request with my PPA added in docs, for people who want latest builds?

I'd like to avoid such instructions and links in the repository.

[ROBERT-MCDOWELL]

yes please thanks much

[i-rinat]

Just released v0.3.6 incorporates mentioned fixes.