search

ppeccin / WebMSX

WebMSX - Online MSX Emulator
https://webmsx.org
396 stars 70 forks source link

LZH decompression of disk images? #36

Open jpoikela opened 6 years ago

jpoikela commented 6 years ago

some archives have files available as LZH compressed disk images, e.g beamride.lzh. on-demand decompression of these disc images would make use of these files a lot more user friendly.

would it be feasible to try to uncompress given LZH image and proceed similar to what is done with ZIP disk images? I have not tested compatibility with MSX files, but at least npm has lzh package available. I have used lhasa to decompress the files and then load the directory as disk to run that kind of images, which is doable, but cumbersome.

jpoikela commented 6 years ago

tried to use npm package "lzh" but it does not compile :unamused:

ppeccin commented 6 years ago

Hi! Yes, I plan to add it, but I have not yet found a good way to do it in javascript.

What you mean it does not compile? Can you direct me to this project? Are you sure it is a complete implementation of the decompression algorithm?

Paulo

On Wed, Jul 11, 2018 at 8:57 AM Jani Poikela notifications@github.com wrote:

tried to use npm package "lzh" but it does not compile 😒

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ppeccin/WebMSX/issues/36#issuecomment-404143227, or mute the thread https://github.com/notifications/unsubscribe-auth/AArloCHWFODzH0aAGp_VXVE4wTjx32baks5uFegsgaJpZM4VFgol .

jpoikela commented 6 years ago

I tried to make a simple node command line with lzh package ( https://www.npmjs.com/package/lzh), to try if it can decompress MSX lzh files.. it has a C-implementation, that does not compile.. didn't have time to look deeper, though.

rewriting lzh in JS could make sense if it was a good implementation (it's very small), but so far I don't know if it is a good one. lhasa is more complete implementation, maybe it should be used for JS rewrite, then..

ke 11. heinäk. 2018 klo 18.29 ppeccin notifications@github.com kirjoitti:

Hi! Yes, I plan to add it, but I have not yet found a good way to do it in javascript.

What you mean it does not compile? Can you direct me to this project? Are you sure it is a complete implementation of the decompression algorithm?

Paulo

On Wed, Jul 11, 2018 at 8:57 AM Jani Poikela notifications@github.com wrote:

tried to use npm package "lzh" but it does not compile 😒

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ppeccin/WebMSX/issues/36#issuecomment-404143227, or mute the thread < https://github.com/notifications/unsubscribe-auth/AArloCHWFODzH0aAGp_VXVE4wTjx32baks5uFegsgaJpZM4VFgol

.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ppeccin/WebMSX/issues/36#issuecomment-404211867, or mute the thread https://github.com/notifications/unsubscribe-auth/AAF5crGcC47vVHZdnM2VpwBtBTUaJuYGks5uFhnKgaJpZM4VFgol .

jpoikela commented 6 years ago

well, the lhasa implementation is not too complex either.. starting at: https://github.com/fragglet/lhasa/blob/master/src/extract.c#L575

jpoikela commented 6 years ago

in a search for an easy solution I made a Kaitai struct https://gist.github.com/jpoikela/f4be749de62bb9716856835829f9a03f that can be tried with https://ide.kaitai.io/# and some lzh file.. seems to get the file parts correctly for my one sample file beamride.lzh, but I only added logic for level 0 and level 1 headers.. level 2 header seemed to be a bit more complex with that tool... and still would need code to decompress the byte arrays...

jpoikela commented 6 years ago

btw, I followed this information about the level 0 and level 1 headers: https://ist.uwaterloo.ca/~schepers/formats/LHA.TXT

jpoikela commented 6 years ago

I try to make a POC with Kaitai, it can be dropped if we get to write stand-alone file parsing..

jpoikela commented 6 years ago

also, running lhasa through emscripten could be worth a POC..

jpoikela commented 6 years ago

well, both seems too cumbersome.. I'll be writing a lhasa-inspired LHZ decoder in JS.

ppeccin commented 6 years ago

Nice! If you can keep it as simple and as light as possible, we can integrate it into WebMSX. If you allow, of course!

Paulo

On Tue, Jul 17, 2018 at 1:13 PM Jani Poikela notifications@github.com wrote:

well, both seems too cumbersome.. I'll be writing a lhasa-inspired LHZ decoder in JS.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ppeccin/WebMSX/issues/36#issuecomment-405639496, or mute the thread https://github.com/notifications/unsubscribe-auth/AArloPnjoDP8nFWwh0n6cKb1HTdauQjQks5uHg0rgaJpZM4VFgol .

jpoikela commented 6 years ago

it's definitely my goal to make it usable for your great WebMSX 🙂

I already wrote some parts, but it may take a week or two to have anything usable - I have not written that much JS.. I might need some help, too. I write it as old-style prototype classes for now. I might modernize it later.

ppeccin commented 6 years ago

Remember, WebMSX is not ES6 yet. So, if possible, try to keep it ES5, or maybe we won't be able to use it.

Also, you should have a look at the ZIP implementation of WebMSX, and how it is used, so you can make your interface as close as possible. You know, the way we must pass the compressed bytes, the way we take the uncompressed files out, etc.

Have fun!

On Thu, Jul 19, 2018 at 6:36 AM Jani Poikela notifications@github.com wrote:

it's definitely my goal to make it usable for your great WebMSX 🙂

I already wrote some parts, but it may take a week or two to have anything usable - I have not written that much JS.. I might need some help, too. I write it as old-style prototype classes for now. I might modernize it later.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ppeccin/WebMSX/issues/36#issuecomment-406217002, or mute the thread https://github.com/notifications/unsubscribe-auth/AArloBxheKJnh9X5lWkAj9-rlCgB7-Gfks5uIFMjgaJpZM4VFgol .

jpoikela commented 6 years ago

hi! I just got first decompression hacked and run in webMSX :sweat_smile: the biggest struggle was with the bit buffer. now, I only have lh0 and lh5 decoders, but I have the impression that I will get some decoders with quite little additional work. the code is a little bit of a hack still, especially the Zip API and integration..

I am a little bit uncomfortable with publishing my code just yet.. I try to make it cleaner first, publish it and the add more decoders.

jpoikela commented 6 years ago

I made a PR, but it's still a work in progress.. still only the lh0 and lh5 decoders.. and not clean code, but a something to try out, if interested.. https://github.com/ppeccin/WebMSX/pull/37

ppeccin commented 6 years ago

Ok! Thank you very much!!!

I need to look at this carefully before moving on. I will let you know... Meanwhile, feel free to make improvements and newer versions while I find the time to look at the code.

Paulo

On Mon, Aug 6, 2018 at 11:16 AM Jani Poikela notifications@github.com wrote:

I made a PR, but it's still a work in progress.. still only the lh0 and lh5 decoders.. and not clean code, but a something to try out, if interested..

37 https://github.com/ppeccin/WebMSX/pull/37

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ppeccin/WebMSX/issues/36#issuecomment-410723159, or mute the thread https://github.com/notifications/unsubscribe-auth/AArloBAVtgTRUxBCPGulzUlpeKf9fOsqks5uOE-8gaJpZM4VFgol .

jpoikela commented 6 years ago

no need to hurry, beamride.lzh seems to be one of the rare cases that actually work..

jpoikela commented 6 years ago

I had time to clean the PR now, so the diff is clean.. sorry for my mistake..

jpoikela commented 6 years ago

I run the embedded version, and load the file like this: ?DISKA_FILES_URL=http://localhost:8000/beamride.lzh&BASIC_BRUN=rider.bin

I have python script serving the files locally

jpoikela commented 6 years ago

I have not found an LZH file that would use other compression method than lh0 or lh5.. still I should include the decoders as soon I find a sample file to test with..

ppeccin commented 6 years ago

Nice. I am still trying to find the time to look at this. For now, I thank you.

On Mon, Aug 13, 2018, 10:45 Jani Poikela notifications@github.com wrote:

I have not found an LZH file that would use other compression method than lh0 or lh5.. still I should include the decoders as soon I find a sample file to test with..

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/ppeccin/WebMSX/issues/36#issuecomment-412522959, or mute the thread https://github.com/notifications/unsubscribe-auth/AArloEbO-VdJ6HJJEEbJEr_-Cdm9t2Vdks5uQYLmgaJpZM4VFgol .