search

guzba / zippy

Pure Nim implementation of deflate, zlib, gzip and zip.
MIT License
246 stars 29 forks source link

[bug] Compression failing when building on a 32-bit machine #3

Closed drygdryg closed 3 years ago

drygdryg commented 3 years ago

It's a simple program that compresses and decompresses a string using gzip, it crashes on 32-bit machines (tested on Arch Linux 32 with Nim 1.4.2):

import zippy
let data = compress("Just for testing", BestCompression, dfGzip)
echo uncompress(data)

Crash log:

$ ./zippy_test                                                                                                     
/home/victor/.nimble/pkgs/zippy-0.5.1/zippy.nim(84) zippy_test
/home/victor/.nimble/pkgs/zippy-0.5.1/zippy.nim(38) compress
/home/victor/.nimble/pkgs/zippy-0.5.1/zippy/deflate.nim(223) deflate
/home/victor/.nimble/pkgs/zippy-0.5.1/zippy/lz77.nim(62) lz77Encode
/usr/lib/nim/system/fatal.nim(49) sysFatal
Error: unhandled exception: value out of range: 2323877985 notin -2147483648 .. 2147483647 [RangeDefect]
guzba commented 3 years ago

Thanks for reporting. I'll need to figure out how to test 32 bit stuff. I think I have a Raspberry Pi that may work.

drygdryg commented 3 years ago

If it is convenient for you, you can create a new branch and send changes there, I will check it out and rebuild zippy.

fwsGonzo commented 3 years ago

Oh wow, I actually got this just now. Too bad I took so long to check the issues! I'm using 32-bit as well!

guzba commented 3 years ago

Hello again. I tried setting up my Pi, however it wouldn't boot even with a fresh image. After that, I installed VirtualBox to try 32 bit Ubuntu. Turns out they do not ship 32 bit images anymore. Downloaded an old Ubuntu image, however VirtualBox error'ed out for me any time I tried to start a VM. Sorry, I think I'm done trying to fight for 32 bit for today. PR welcome though if you have 32 bit OS running somehow!

drygdryg commented 3 years ago

You can build a 32-bit executable on a 64-bit system by passing the -m32 flag to the C compiler. Here's an example (I used GCC 10.2 on Arch Linux): zippy_test.nim contents:

import zippy
let data = compress("Just for testing", BestCompression, dfGzip)
echo uncompress(data)

I built it with: nim c --cpu:i386 --passC:"-m32" --passL:"-m32" zippy_test.nim Run:

$ ./zippy_test 
/home/victor/.nimble/pkgs/zippy-0.5.1/zippy.nim(84) zippy_test
/home/victor/.nimble/pkgs/zippy-0.5.1/zippy.nim(38) compress
/home/victor/.nimble/pkgs/zippy-0.5.1/zippy/deflate.nim(223) deflate
/home/victor/.nimble/pkgs/zippy-0.5.1/zippy/lz77.nim(62) lz77Encode
/usr/lib/nim/system/fatal.nim(49) sysFatal
Error: unhandled exception: value out of range: 2323877985 notin -2147483648 .. 2147483647 [RangeDefect]

Please note that your distribution must contain 32-bit libraries. For example, on Debian/Ubuntu, you can install them like this: sudo apt-get install gсс-multilib libc6-dev-i386 On Arch Linux: 1) enable multilib repo: https://wiki.archlinux.org/index.php/Official_repositories#multilib 2) install 32-bit libraries: sudo pacman -S lib32-gcc-libs lib32-glibc

guzba commented 3 years ago

Ah, that looked so promising. Sadly the build errors out for me. I dev on Windows right now to ensure it works there (since it is so often ignored) and i have 64 bit mingw installed, guessing that is related.

I did get my Pi to boot though so that's good. Just needed to unplug the mouse and keyboard until after it booted: https://raspberrypi.stackexchange.com/questions/107200/fresh-installed-image-not-booting I have no words lol.

Hoping I can get Nim up and running and not only over 32 bit but also ARM stuff.

guzba commented 3 years ago

Ok! good news. I was able to get Nim built on my Pi and successfully reproduce the issue. Fixed it and tagged a new release with the fix as version 0.5.2.

I'll be doing future releases that improve performance for 32 bit (there are opportunities here). In the meantime zippy should work now.

drygdryg commented 3 years ago

Thank you!