XZ Utils (LZMA2) doesn't properly work via emscripten javascript

[yurevich1]

I'm trying to compile js using XZ Utils (http://tukaani.org/xz/ - git clone http://git.tukaani.org/xz.git ) library. OS Debian. I'f I compile using default gcc-make pipe-line all works correctly: setup: git clone http://git.tukaani.org/xz.git cd xz ./autogen.sh ./configure make cd doc/examples gcc 01_compress_easy.c -I../../src/liblzma/api -L../../src/liblzma/.libs/liblzma.a ../../src/liblzma/.libs/liblzma.so -o 01_comp echo "All your base are belong to us" > a.txt cat a.txt | ./02_comp 7 > a.txt.xz This algorithm creates correct archive.

But if I use emscripten:

../emscripten/emcofigure ./autogen.sh ../emscripten/emcofigure ./configure ../emscripten/emmake make cd doc/examples ../../../emscripten/emcc 01_compress_easy_modified.c -I../../src/liblzma/api -L../../src/liblzma/.libs/liblzma.a ../../src/liblzma/.libs/liblzma.so -o 01_comp.js Inside 01_compress_easy_modified.c I'm creating "filesystem" with input and output files and show bytes via

while ((nread = fread(buf, 1, sizeof buf, file_out)) > 0) { fprintf(stdout,"\n%d",nread); for(int i = 0; i < nread; i++) fprintf(stdout," 0x%x",buf[i] & 0xff); }

And it show different bytes which the XZ Utils cannot decompress.

But if I try to decompress the correct bytes all works perfect. I'm really don't where is a mistake. Please, help to fix the error.

[kripken]

Take a look at http://kripken.github.io/emscripten-site/docs/porting/Debugging.html#compiler-settings , especially SAFE_HEAP etc.

[yurevich1]

Flags:

emscripten/emcc -DHAVE_CONFIG_H -I. -I../.. -DLOCALEDIR=\"/usr/local/share/locale\" -I../../src/common -I../../src/liblzma/api -I../../lib -fvisibility=hidden -Wall -Wextra -Wvla -Wformat=2 -Winit-self -Wmissing-include-dirs -Wstrict-aliasing -Wfloat-equal -Wundef -Wshadow -Wpointer-arith -Wbad-function-cast -Wwrite-strings -Waggregate-return -Wstrict-prototypes -Wold-style-definition -Wmissing-prototypes -Wmissing-declarations -Wmissing-noreturn -Wredundant-decls -s SAFE_HEAP=1 -MT xz-main.o -MD -MP -MF

the results.

alignment error loading from address 5590905, which was expected to be aligned to a multiple of 2 alignment error loading from address 5590905, which was expected to be aligned to a multiple of 2

../nodejs/sourse/xz/emsc/fs.js:90 throw ex; ^ abort("alignment error loading from address 5590905, which was expected to be aligned to a multiple of 2") at Error at jsStackTrace (../nodejs/sourse/xz/emsc/fs.js:1229:13) at stackTrace (../nodejs/sourse/xz/emsc/fs.js:1246:22) at abort (../nodejs/sourse/xz/emsc/fs.js:59550:44) at SAFE_HEAP_LOAD (../nodejs/sourse/xz/emsc/fs.js:359:27) at _lzma_lzma_optimum_fast (../nodejs/sourse/xz/emsc/fs.js:27853:9) at _lzma_lzma_encode (../nodejs/sourse/xz/emsc/fs.js:43300:4) at Array._lzma2_encode as 63 at Array._lz_encode as 79 at _block_encode_normal (../nodejs/sourse/xz/emsc/fs.js:41563:78) at _block_buffer_encode (../nodejs/sourse/xz/emsc/fs.js:34247:14)

[yurevich1]

I've found the 'bug' places:

1. I've commented lines in file: lzma_encoder_private.h //#ifdef TUKLIB_FAST_UNALIGNED_ACCESS // #define not_equal_16(a, b) \ ((const uint16_t )(a) != (const uint16_t )(b)) //#else

define not_equal_16(a, b) \

    ((a)[0] != (b)[0] || (a)[1] != (b)[1])

//#endif

1. In file memcmplen.h in fuction lzma_memcmplen I've commented all exapt

   define LZMA_MEMCMPLEN_EXTRA 0

   while (len < limit && buf1[len] == buf2[len]) ++len; return len;

I think the errors appear about (const uint16_t )(a) and (const uint32_t )(buf1 + len) conversions .

Thanks. -s SAVE_HEAP=1 is very usefull feature. But it is very difficult to find it. Please, write more obviously in manual. It is difficult to find this feature.

[kripken]

If you can think of a way to improve the docs, a pull request with that would be very welcome. Perhaps try to think about where the best place for you would have been, and write something there.

[yurevich1]

OK, when I'll have a fre time I'll try to help to impreve the manual. the other problem: if I set

I receive in the Browser (Chrome ) User Agent : Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/43.0.2357.134 Safari/537.36 Invalid array buffer length var buffer; buffer = new ArrayBuffer(TOTAL_MEMORY);

if TOTAL_MEMORY=2147483648

In my case I cannot use memory more than 1GB. But algorithm wants.

Is this a browser restriction ? Or I can prevent to avoid this error by some flags (etc.)?

[kripken]

Yes, most browsers have problems allocating 1GB or more of continuous memory for typed arrays, I'm afraid. This is improving, but still is not reliable yet.

[stale[bot]]

This issue has been automatically marked as stale because there has been no activity in the past 2 years. It will be closed automatically if no further activity occurs in the next 7 days. Feel free to re-open at any time if this issue is still relevant.