search

Godzil / theunarchiver

Automatically exported from code.google.com/p/theunarchiver
Other
0 stars 0 forks source link

Improve DiskDoubler support #149

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago 
The version of Unarchiver I just downloaded does not seem to decompress
files compressed by DiskDoubler 3.7.7. It returns a message with "Unknown
data format". if one continues, a blank decompressed file is produced.

I am running Mac OSX 10.5.6. I see that the Preferences includes a check
box for

DiskDoubler Archive extensions dd, DD.

Is there something I am doing wrong or does Unarchiver not decompress
DiskDoubler 3.7.7 files? If so, any suggestions on how to decompress these
files?

MANY thanks,

S.

Original issue reported on code.google.com by orz...@freshpond.org on 25 Mar 2009 at 6:02

GoogleCodeExporter commented 8 years ago 
DiskDoubler uses a whole lot of different algorithms, and only a few are 
supported. A couple more have been 
added for 2.0, but many are still missing.

Once again, nothing can be done without test case files, and although I have a 
few more are always useful.

Original comment by paracel...@gmail.com on 1 Apr 2009 at 11:22

GoogleCodeExporter commented 8 years ago 
[deleted comment]
GoogleCodeExporter commented 8 years ago 
[deleted comment]
GoogleCodeExporter commented 8 years ago 
OK, I can confirm that extraction of DiskDoubler archive with 
TheUnarchiver2.0alpha3 fails ... here's the sample 
file.

Original comment by koo...@gmail.com on 20 Jun 2009 at 3:41

Attachments:

GoogleCodeExporter commented 8 years ago 
Issue 187 has been merged into this issue.

Original comment by paracel...@gmail.com on 24 Aug 2009 at 9:26

GoogleCodeExporter commented 8 years ago 
Issue 223 has been merged into this issue.

Original comment by paracel...@gmail.com on 6 Dec 2009 at 2:46

GoogleCodeExporter commented 8 years ago 
Reverse-engineered and added support for the AD and DD algorithms in 
DiskDoubler 
(methods 6, 9 and 10). With this, I can unpack all files I have available. I am 
still missing 
methods 2 and 5 (same algorithm, different settings), but as I have no test 
files for these 
I can't do anything about it yet.

If anyone has a file that won't open in 2.3 or newer, please post it here! For 
now, I'm 
going to assume most DiskDoubler files will now unpack correctly.

Original comment by paracel...@gmail.com on 8 Jan 2010 at 2:53

GoogleCodeExporter commented 8 years ago 
Hello,
I tested the unarchiver these days to decompress some old DD files.
In many cases the app return the error: Couldn't extract the file ...
The archives contains singularly TIFF or EPS (DCS) files and it seems that only 
the original TIFF files are 
extracted.
I attach here five DCS files that return error (together are used to compone a 
single EPS file).
Please try with these file if you can find the missing methods.
Thanks for your work.
Sorry for my bad English.

Kind Regards
Marco

Original comment by mac.marc...@gmail.com on 9 Apr 2010 at 7:50

Attachments:

GoogleCodeExporter commented 8 years ago 
Oh, these seem to use the DiskDoubler delta preprocessing. This should be easy 
to add support for, but I haven't 
found any test files for it so far. These should help a lot! Thanks!

Original comment by paracel...@gmail.com on 9 Apr 2010 at 10:31

GoogleCodeExporter commented 8 years ago 
I added an implementation of the delta preprocessing. Unfortunately, this 
compression method does not store a 
checksum for the file, so I can't tell if it is 100% correct. It looks sort of 
right, though, so I suspect it works.

If you could test it when the next version is released and tell me if it works 
right, that would help.

Original comment by paracel...@gmail.com on 9 Apr 2010 at 9:15

GoogleCodeExporter commented 8 years ago 
Ok! Thank you very much for your (rapid) work.
I'm sure I'll test the new version. Do you know when about it will be released?

Original comment by mac.marc...@gmail.com on 12 Apr 2010 at 7:03

GoogleCodeExporter commented 8 years ago 
No, but I'm hoping for "soon".

Original comment by paracel...@gmail.com on 12 Apr 2010 at 11:59

GoogleCodeExporter commented 8 years ago 
Sorry for the OT, but I see just now that there is no Italian language for The 
Unarchiver.
I use XCode and IB for hobby and I know how to use them. So I could translate 
English.lproj in Italian.
But how can I download only the English.lproj folder and then upload 
Italian.lproj to add it to the next release of 
The Unarchiver?
I attend your answer, thank you!

Original comment by mac.marc...@gmail.com on 13 Apr 2010 at 9:01

GoogleCodeExporter commented 8 years ago 
Download the source archive, or get the source from SVN, and then open a new 
issue and upload the Italian.lproj 
there.

Original comment by paracel...@gmail.com on 13 Apr 2010 at 10:32

GoogleCodeExporter commented 8 years ago 
Attached are two Quark files from 1998 archived using DiskDoubler. I've got an 
older version of the original 
application working on a Mac G4. When I use the original app they decompress 
fine but when I use unarchiver I 
get a data corrupt message. Some files from 2005 decompress fine. Perhaps the 
attached were compressed with 
a much older version.

Original comment by penicuik...@gmail.com on 13 Apr 2010 at 6:23

Attachments:

GoogleCodeExporter commented 8 years ago 
@penicuikboy:
I noted that some of my old DD files return data corrupt message, but if I 
continue then the original files are 
correctly extracted.
Instead when it return "Couldn't extract the file" message the files are really 
not extracted.
Try to open it likewise.
Let us know if it works.

Original comment by mac.marc...@gmail.com on 13 Apr 2010 at 6:49

GoogleCodeExporter commented 8 years ago 
Issue 247 has been merged into this issue.

Original comment by paracel...@gmail.com on 14 Apr 2010 at 10:33

GoogleCodeExporter commented 8 years ago 
Fixed the extraction of the files in comment 15. The DiskDoubler compressor 
seems to emit one byte too much 
for some files, and that has to be handled. I'd fixed it for AD compression but 
not DD.

Original comment by paracel...@gmail.com on 17 Apr 2010 at 6:46

GoogleCodeExporter commented 8 years ago 
Issue 250 has been merged into this issue.

Original comment by paracel...@gmail.com on 23 Apr 2010 at 3:51

GoogleCodeExporter commented 8 years ago 
Unable to extract these files
DiskDoubler ca. 1994
Can you help?

Original comment by ithaca...@gmail.com on 30 Apr 2010 at 6:59

Attachments:

GoogleCodeExporter commented 8 years ago 
Hmm, I see nothing strange about those files. They should extract with any 
recent version of The Unarchiver. 
What exact problem are you having, and with what version?

Original comment by paracel...@gmail.com on 30 Apr 2010 at 7:11

GoogleCodeExporter commented 8 years ago 
Would you mind a 16.2 MB upload? It doesn't quite decompress completely. This 
is a 1996 file. It throws an error  
on decrunching and images don't make it out of the file.

Original comment by jelmore...@gmail.com on 20 May 2010 at 3:14

GoogleCodeExporter commented 8 years ago 
If Google accepts it, upload away.

Original comment by paracel...@gmail.com on 20 May 2010 at 10:58

GoogleCodeExporter commented 8 years ago 
Well, Google's limit is 10.0 MB. You could download from here if you like:
http://elmoredesign.com/Bum_Equipment.dd

Original comment by jelmore...@gmail.com on 20 May 2010 at 12:19

GoogleCodeExporter commented 8 years ago 
Got it, thanks. It does fail here too, but I am not sure if it is a bug or if 
the file 
is corrupted. I'll look into it.

Do you know if the file can be unpacked with DiskDoubler itself?

Original comment by paracel...@gmail.com on 20 May 2010 at 12:43

GoogleCodeExporter commented 8 years ago 
> Do you know if the file can be unpacked with DiskDoubler itself?

I don't have any way of doing that. Seems a long time ago I got some files out 
of it but I don't really recall.

Original comment by jelmore...@gmail.com on 20 May 2010 at 2:32

GoogleCodeExporter commented 8 years ago 
> Do you know if the file can be unpacked with DiskDoubler itself?
Hi,
I have DiskDoubler in a old Mac.
I've tested it with the file but it got an error saying the file seems 
corrupted.

Original comment by mac.marc...@gmail.com on 20 May 2010 at 3:15

GoogleCodeExporter commented 8 years ago 
Well dang. Thanks you all. That file probably came off of a recovered optical 
drive disc. That was back in the 
days of a $1200 drive with $125 132 MB discs. Man that was expensive.

Original comment by jelmore...@gmail.com on 20 May 2010 at 3:24

GoogleCodeExporter commented 8 years ago 
Attached is a file compressed using the Sigma accelerator card algorithm (which 
used a chip from Stac). This appears to be unimplemented.

Original comment by d235...@gmail.com on 29 Sep 2010 at 3:18

Attachments:

GoogleCodeExporter commented 8 years ago 
Interesting. Is the algorithm documented anywhere?

Original comment by paracel...@gmail.com on 29 Sep 2010 at 3:20

GoogleCodeExporter commented 8 years ago 
Seems it was used elsewhere too and is simple and documented. I'll have to have 
a look, it might be easy to support.

Original comment by paracel...@gmail.com on 29 Sep 2010 at 3:26

GoogleCodeExporter commented 8 years ago 
It's hard to say, since this was by Stac, it may have been used in Stacker and 
possibly Microsoft DoubleSpace (which they ripped off from Stac).

Original comment by d235...@gmail.com on 29 Sep 2010 at 4:30

GoogleCodeExporter commented 8 years ago 
Perhaps it's just LZS: http://en.wikipedia.org/wiki/Lempel–Ziv–Stac

Original comment by d235...@gmail.com on 29 Sep 2010 at 8:04

GoogleCodeExporter commented 8 years ago 
I suspect it is LZS. The Sigma cards supposed used the Stac 9703 chip, and 
according to a paper I found, the Stac 9703 should implement LZS.

However, the data in the file is not valid LZS. DiskDoubler is kind of 
notorious for obfuscating its data in different ways, so it's probably just 
that. Haven't figured out how it's working yet, though. The ancient macutils 
package has an implementation for this data format, but it is marked as 
"untested" and does not seem to actually work either.

Do you have any more files in this format? And do you have the unpacked 
versions of any of them? Both would be very useful.

Original comment by paracel...@gmail.com on 29 Sep 2010 at 9:33

GoogleCodeExporter commented 8 years ago 
I have a card, and it's in a working Mac. Compression with the card is over ten 
times faster than with software.

If you provide me with an archive of test pattern files, I can compress each 
one.

Also I have some .SIT files which The Unarchiver chokes on, as well as some 
self-extracting archives of various types. I'll eventually open tickets for 
those.

Original comment by d235...@gmail.com on 29 Sep 2010 at 9:59

GoogleCodeExporter commented 8 years ago 
Oh, that is interesting. I put together eight fields of various complexity to 
run through it. The last ones are a little big, if that turns into a problem 
then ignore them.

If you have an old version of DD running, could you also see if you can get it 
to produce files with other algorithms? Currently I am lacking test cases for 
methods 2, 3, 4 and 5. No idea what DD might call those, but 2 and 5 are 
variations on the same one, and 3 and 4 might be an RLE algoithm used by 
PackIt,  and some kind of Huffman algorithm.

Also looking forward to any test cases for anything else that isn't working 
yet. Can't guarantee I'll get it all working, but without files I can't even 
try.

Original comment by paracel...@gmail.com on 29 Sep 2010 at 10:29

Attachments:

GoogleCodeExporter commented 8 years ago 
I'm looking for old versions of DD.

The chip on the card is a Stac 9703PC4.

I'll run the testcases as soon as possible. Size isn't an issue, as long as it 
fits on a Zip disk :)

Original comment by d235...@gmail.com on 29 Sep 2010 at 10:31

GoogleCodeExporter commented 8 years ago 
A document I found claims that what you're calling DD B was originally DD C and 
DD B had been dropped. I'll see if I can find the document. Ideally, I'd need 
to find an older version of DD.

Original comment by d235...@gmail.com on 29 Sep 2010 at 10:40

GoogleCodeExporter commented 8 years ago 
Well, I finally made some progress: It is LZS, but with some kind of extra 
header, and with all bytes inverted before and after compression for no good 
reason. I think it may also be separating the data into blocks.

I'll keep hacking at it. The test cases will still be useful to confirm the 
code really works, once I get that far.

Original comment by paracel...@gmail.com on 30 Sep 2010 at 12:24

GoogleCodeExporter commented 8 years ago 
Here are the test cases.

Original comment by d235...@gmail.com on 30 Sep 2010 at 12:30

Attachments:

GoogleCodeExporter commented 8 years ago 
Attached are testcases for DD 1.0. It was impossible to use the testcases you 
provided for 1.0 Method 2, because Disk Doubler 1.0 tries both methods and uses 
whichever produces the smaller file.

Original comment by d235...@gmail.com on 30 Sep 2010 at 4:15

Attachments:

GoogleCodeExporter commented 8 years ago 
Attached are testcases for DD 3.0.1. This one allowed selection of method A or 
B.

Original comment by d235...@gmail.com on 30 Sep 2010 at 4:17

Attachments:

GoogleCodeExporter commented 8 years ago 
Attached are testcases for DD 3.1. These are probably of the same type as DD 
3.0.1.

Original comment by d235...@gmail.com on 30 Sep 2010 at 4:19

Attachments:

GoogleCodeExporter commented 8 years ago 
Attached are testcases for DD 3.7.2. I also have 3.7.7 but I'm quite sure it 
will generate the same format data.

The Sigma card output is from 3.7.7.

DD 4.0 formats (DD 1, DD 2, and DD 3) are well supported. AD 1 and 2 seem to 
be, though I haven't been able to test either.

Original comment by d235...@gmail.com on 30 Sep 2010 at 4:21

Attachments:

GoogleCodeExporter commented 8 years ago 
Ok, I think I have code that handles LZS now. The checksumming is pretty 
limited for this format (just a XOR of all bytes) but all the files I have pass 
it now.

Code for that is checked in, you'd have to build it yourself if you want to 
test, though.

Going to look at the other samples next.

Original comment by paracel...@gmail.com on 30 Sep 2010 at 12:24

GoogleCodeExporter commented 8 years ago 
Also, from the other samples, I've so far gathered three things: There's a bug 
in recognizing really old DiskDoubler files, which I fixed. There's also a bug 
in unpacking really big Compress files, which I still need to fix. And there 
are some method 2 files in there, which will be useful if I manage to reverse 
engineer the format (I started on that some time back, but didn't bother going 
very far since I lacked examples).

Still missing methods 3, 4 and 5 (unless some of those are in here and I missed 
them due to crashes). 5 should be the same as 2 with a different parameter, but 
3 and 4 are entirely different. I think they might be very bad but fast 
algorithms, so maybe only old AutoDoubler uses them, or something? Not sure 
exactly how one would get a hold of those.

Original comment by paracel...@gmail.com on 30 Sep 2010 at 12:34

GoogleCodeExporter commented 8 years ago 
I will produce some AutoDoubler samples later today. I don't think I have AD 
1.0 but I do have several older versions.

It would be nice if the format of Disk Copy 6 image compression was reversed. 
This is quite annoying since it uses a 'bcem' resource to store what appears to 
be the compression  dictionaries. A simple NDIF Compressed to NDIF converter 
would be quite useful. ShrinkWrap is another format that uses these 'bcem' 
resources but with its own algorithms (which Disk Utility cannot handle). Just 
a thought.

Original comment by d235...@gmail.com on 30 Sep 2010 at 1:22

GoogleCodeExporter commented 8 years ago 
l'll file a ticket if you think it's worthwhile.

Original comment by d235...@gmail.com on 30 Sep 2010 at 1:25

GoogleCodeExporter commented 8 years ago 
Ideally I'd like to support any format that has been in actual use, so filing a 
bug is probably a good idea. However, the main problem is that the entire 
architecture of the app isn't designed to handle files that actually use 
multiple forks for their compression data. I'd have to figure out some way to 
deal with that, which I don't think it would be very high priority. But it's 
still good to have the files and the idea here for future reference.

Original comment by paracel...@gmail.com on 30 Sep 2010 at 1:30

GoogleCodeExporter commented 8 years ago 
AutoDoubler may have used those weird formats to compress resources within 
extensions and control panels. I shall do further investigation.

Original comment by d235...@gmail.com on 30 Sep 2010 at 3:42