search

richgel999 / lzham_codec

Lossless data compression codec with LZMA-like ratios but 1.5x-8x faster decompression speed, C/C++
Other
693 stars 71 forks source link

lzham in JavaScript and Python? #9

Open jedie opened 9 years ago

jedie commented 9 years ago

Is there a implementation in JavaScript and Python ?

richgel999 commented 9 years ago

You can compile LZHAM to Javascript using emscripten (which I believe has already been done internally by some folks at Mozilla). One of my other C++ codecs (crunch) has been compiled successfully with emscripten, so I know it's possible.

jedie commented 9 years ago

I have tried to compile. Without success :(

Any tutorial / howto out there?!?

jedie commented 9 years ago

Here the steps i tried:

~ $ sudo apt-get install build-essential cmake nodejs
~ $ cd ~/emsdk_portable

~/emsdk_portable $ ./emsdk update
~/emsdk_portable $ ./emsdk install latest
~/emsdk_portable $ ./emsdk activate latest

~/emsdk_portable $ emcc -v
INFO     root: (Emscripten: Running sanity checks)
emcc (Emscripten gcc/clang-like replacement) 1.33.0
clang version 3.7.0 (https://github.com/kripken/emscripten-fastcomp-clang/ 2a66b5ffba63f1e36458d41fafc2874a1abf63fe) (https://github.com/kripken/emscripten-fastcomp/ e2939f6878307899c17d1a23f51ba3fe5afbe6af)
Target: i686-pc-linux-gnu
Thread model: posix
Found candidate GCC installation: /usr/lib/gcc/i686-linux-gnu/4.8
Found candidate GCC installation: /usr/lib/gcc/i686-linux-gnu/4.8.2
Found candidate GCC installation: /usr/lib/gcc/i686-linux-gnu/4.9
Found candidate GCC installation: /usr/lib/gcc/i686-linux-gnu/4.9.1
Selected GCC installation: /usr/lib/gcc/i686-linux-gnu/4.8
Candidate multilib: .;@m32
Selected multilib: .;@m32
INFO     root: (Emscripten: Running sanity checks)

~/emsdk_portable $ git clone https://github.com/richgel999/lzham_codec.git
~/emsdk_portable $ cd lzham_codec
~/emsdk_portable/lzham_codec $ mkdir build
~/emsdk_portable/lzham_codec $ cd build
~/emsdk_portable/lzham_codec/build $ cmake ..
...
-- Build files have been written to: ~/emsdk_portable/lzham_codec/build

~/emsdk_portable/lzham_codec/build $ make
Scanning dependencies of target lzhamdecomp
[  5%] Building CXX object lzhamdecomp/CMakeFiles/lzhamdecomp.dir/lzham_assert.cpp.o
[ 10%] Building CXX object lzhamdecomp/CMakeFiles/lzhamdecomp.dir/lzham_checksum.cpp.o
[ 15%] Building CXX object lzhamdecomp/CMakeFiles/lzhamdecomp.dir/lzham_huffman_codes.cpp.o
[ 20%] Building CXX object lzhamdecomp/CMakeFiles/lzhamdecomp.dir/lzham_lzdecompbase.cpp.o
[ 25%] Building CXX object lzhamdecomp/CMakeFiles/lzhamdecomp.dir/lzham_lzdecomp.cpp.o
[ 30%] Building CXX object lzhamdecomp/CMakeFiles/lzhamdecomp.dir/lzham_mem.cpp.o
[ 35%] Building CXX object lzhamdecomp/CMakeFiles/lzhamdecomp.dir/lzham_platform.cpp.o
[ 40%] Building CXX object lzhamdecomp/CMakeFiles/lzhamdecomp.dir/lzham_prefix_coding.cpp.o
[ 45%] Building CXX object lzhamdecomp/CMakeFiles/lzhamdecomp.dir/lzham_symbol_codec.cpp.o
[ 50%] Building CXX object lzhamdecomp/CMakeFiles/lzhamdecomp.dir/lzham_timer.cpp.o
[ 55%] Building CXX object lzhamdecomp/CMakeFiles/lzhamdecomp.dir/lzham_vector.cpp.o
Linking CXX shared library liblzhamdecomp.so
[ 55%] Built target lzhamdecomp
Scanning dependencies of target lzhamcomp
[ 60%] Building CXX object lzhamcomp/CMakeFiles/lzhamcomp.dir/lzham_lzbase.cpp.o
[ 65%] Building CXX object lzhamcomp/CMakeFiles/lzhamcomp.dir/lzham_lzcomp.cpp.o
[ 70%] Building CXX object lzhamcomp/CMakeFiles/lzhamcomp.dir/lzham_lzcomp_internal.cpp.o
[ 75%] Building CXX object lzhamcomp/CMakeFiles/lzhamcomp.dir/lzham_lzcomp_state.cpp.o
[ 80%] Building CXX object lzhamcomp/CMakeFiles/lzhamcomp.dir/lzham_match_accel.cpp.o
[ 85%] Building CXX object lzhamcomp/CMakeFiles/lzhamcomp.dir/lzham_pthreads_threading.cpp.o
Linking CXX shared library liblzhamcomp.so
[ 85%] Built target lzhamcomp
Scanning dependencies of target lzhamdll
[ 90%] Building CXX object lzhamdll/CMakeFiles/lzhamdll.dir/lzham_api.cpp.o
Linking CXX shared library liblzhamdll.so
[ 90%] Built target lzhamdll
Scanning dependencies of target lzhamtest
[ 95%] Building CXX object lzhamtest/CMakeFiles/lzhamtest.dir/timer.cpp.o
[100%] Building CXX object lzhamtest/CMakeFiles/lzhamtest.dir/lzhamtest.cpp.o
Linking CXX executable lzhamtest
[100%] Built target lzhamtest

~/emsdk_portable/lzham_codec/build $ emmake make
[ 55%] Built target lzhamdecomp
[ 85%] Built target lzhamcomp
[ 90%] Built target lzhamdll
[100%] Built target lzhamtest

+ cd lzham_codec
+ cd build
+ emcc -O3 -I../include/ -I../lzhamdll/ ./lzhamdll/liblzhamdll.so ./lzhamdecomp/liblzhamdecomp.so ./lzhamcomp/liblzhamcomp.so -o lzham.js
WARNING  root: emcc: cannot find library "lzhamdll"
WARNING  root: emcc: cannot find library "lzhamdecomp"
WARNING  root: emcc: cannot find library "lzhamcomp"
ERROR    root: no input files
note that input files without a known suffix are ignored, make sure your input files end with one of: ('.c', '.C', '.i', '.cpp', '.cxx', '.cc', '.c++', '.CPP', '.CXX', '.CC', '.C++', '.ii', '.m', '.mi', '.mm', '.mii', '.bc', '.o', '.obj', '.dylib', '.so', '.a', '.ll', '.h', '.hxx', '.hpp', '.hh', '.H', '.HXX', '.HPP', '.HH')

But the files are there:

    ~/emsdk_portable $ ls -la lzham_codec/build/lzhamdll
    insgesamt 44
    drwxr-xr-x 3 jens jens  4096 Jun 16 09:11 .
    drwxr-xr-x 7 jens jens  4096 Jun 16 09:11 ..
    drwxr-xr-x 3 jens jens  4096 Jun 16 09:11 CMakeFiles
    -rw-r--r-- 1 jens jens  2213 Jun 16 09:01 cmake_install.cmake
    -rwxr-xr-x 1 jens jens 18194 Jun 16 09:01 liblzhamdll.so
    -rw-r--r-- 1 jens jens  7213 Jun 16 09:01 Makefile

    ~/emsdk_portable $ ls -la lzham_codec/build/lzhamdecomp
    insgesamt 164
    drwxr-xr-x 3 jens jens   4096 Jun 16 09:11 .
    drwxr-xr-x 7 jens jens   4096 Jun 16 09:11 ..
    drwxr-xr-x 3 jens jens   4096 Jun 16 09:11 CMakeFiles
    -rw-r--r-- 1 jens jens   2145 Jun 16 09:01 cmake_install.cmake
    -rwxr-xr-x 1 jens jens 129201 Jun 16 09:01 liblzhamdecomp.so
    -rw-r--r-- 1 jens jens  18286 Jun 16 09:01 Makefile

    ~/emsdk_portable $ ls -la lzham_codec/build/lzhamcomp
    insgesamt 152
    drwxr-xr-x 3 jens jens   4096 Jun 16 09:11 .
    drwxr-xr-x 7 jens jens   4096 Jun 16 09:11 ..
    drwxr-xr-x 3 jens jens   4096 Jun 16 09:11 CMakeFiles
    -rw-r--r-- 1 jens jens   2223 Jun 16 09:01 cmake_install.cmake
    -rwxr-xr-x 1 jens jens 122512 Jun 16 09:01 liblzhamcomp.so
    -rw-r--r-- 1 jens jens  13022 Jun 16 09:01 Makefile

Any idea?

hackcasual commented 9 years ago

Use make, not emmake

JavaScript-Packer commented 9 years ago

Would love a compress/decompress functions for JavaScript to share with the WOrld

jeanbmar commented 6 years ago

This is how I compiled successfully on Debian:

  1. Clone https://github.com/richgel999/lzham_codec.git

  2. Edit lzham_codec/lzhamdecomp/lzham_types.h and change:

    #if defined(__APPLE__) || defined(__FreeBSD__) || defined(__NetBSD__)
#undef INT8_MIN
#undef INT8_MAX
#undef UINT8_MIN
#undef UINT8_MAX

#undef INT16_MIN
#undef INT16_MAX
#undef UINT16_MIN
#undef UINT16_MAX

#undef INT32_MIN
#undef INT32_MAX
#undef UINT32_MIN
#undef UINT32_MAX

#undef INT64_MIN
#undef INT64_MAX
#undef UINT64_MIN
#undef UINT64_MAX
#endif

    to:

    
#undef INT8_MIN
#undef INT8_MAX
#undef UINT8_MIN
#undef UINT8_MAX

undef INT16_MIN

undef INT16_MAX

undef UINT16_MIN

undef UINT16_MAX

undef INT32_MIN

undef INT32_MAX

undef UINT32_MIN

undef UINT32_MAX

undef INT64_MIN

undef INT64_MAX

undef UINT64_MIN

undef UINT64_MAX

3. Run the following commands:

cd lzham_codec mkdir build cd build emcmake cmake .. emmake make em++ -O3 ./lzhamdll/liblzhamdll.so ./lzhamdecomp/liblzhamdecomp.so ./lzhamcomp/liblzhamcomp.so -o lzham.js