Support for RAR

[GoogleCodeExporter]

Hi,

it would be great if libarchive supported the RAR format; even if it would
be passworded archive.

This is just a wish/enhancement, not a bug; and I know it's probably not
easy to implement and may take a long time. And thanks for this great
project, I use it through GNOME's gvfs-mount.

Original issue reported on code.google.com by ondra.pe...@gmail.com on 5 Oct 2009 at 4:05

• Blocked on: #166

[GoogleCodeExporter]

I'm pretty sure that won't happen. The RAR format decoders are free-as-in-beer 
but
not free-as-in-speech AND if you use the "free" decoder you are agreeing never 
to try
to reverse-engineer and create your own encoder.  Needless to say (but I'll say 
it
anyway) the RARlabs encoders are not free in any sense.

Now, libarchive is BSDish, not GPLish, so maybe that's of less concern to Tim...

Original comment by cwils...@gmail.com on 6 Oct 2009 at 2:10

[GoogleCodeExporter]

The deeper problem is that the existing RAR code almost certainly will not 
integrate
cleanly into libarchive.  So RAR support would probably require at least 
refactoring
the existing RAR code and may require rewriting parts of it more-or-less from
scratch.  This hasn't been a problem for well-documented open formats such as 
ZIP and
ISO9660 but could be a problem for RAR.

My suggestion:  Talk to the RAR developers and ask them if they would help with 
such
a project.  If they don't want to build a libarchive module themselves 
(libarchive's
internal API is pretty stable at this point, so this could be a reasonable 
approach),
they may be willing to provide enough documentation for someone else to 
implement
such a module.  It never hurts to ask!

Note: The distribution of such a module would also be a little tricky.  I've 
tried to
keep the libarchive distribution entirely BSD-licensed.  If the RAR developers 
insist
on a more restrictive license, they could either distribute their own fork of
libarchive or I could try to come up with a way to organize the libarchive
distribution so that there was a well-defined area for code with different 
licenses.
 I have no objection at all to someone distributing their own fork and would be
entirely open to discussing the latter approach.

Original comment by kientzle@gmail.com on 6 Oct 2009 at 3:14

[GoogleCodeExporter]

I knew it won't be easy... :)
the idea came from VLC, that's capable of playing movies inside RAR archive(s), 
and
my wish of Totem being able to do this too. One way of doing this was through
mounting the RAR(s) with gvfs (with libarchive) and than playing it. The other 
way is
writing a special plugin for GStreamer. Both ways are pretty difficult... I will
probably try to takl to GStreamer guys too.
Anyway, I will contact RARlabs and track the progress of the negotiation here...

Original comment by ondra.pe...@gmail.com on 6 Oct 2009 at 10:24

[GoogleCodeExporter]

reply:
Eugene Roshal

Hello,

Sorry, we do not plan to make such libarchive module and we do not
have any additional documentation about RAR format. All what we have
now is the brief archive headers description in technote.txt
and unrar source code available in "RAR extras" on www.rarlab.com.

If libarchive team will decide to make such module, I can answer to
their questions related to any unclear parts of unrar source code
by email.

Eugene

Original comment by ondra.pe...@gmail.com on 9 Oct 2009 at 10:28

[GoogleCodeExporter]

package containing "technote.txt": 
http://www.rarlab.com/rar/rarlinux-3.9.0.tar.gz
unrar source code: http://www.rarlab.com/rar/unrarsrc-3.9.6.tar.gz
email: roshal(at)rarlab.com

Original comment by ondra.pe...@gmail.com on 9 Oct 2009 at 10:45

[GoogleCodeExporter]

The unrar sources are not useful.  Because of the unusual license restrictions 
on
them, we cannot use any part of them in libarchive.  In fact, anyone who has 
read
them carefully cannot be allowed to work on RAR support for libarchive.  (I 
myself
downloaded, skimmed the readme, technote, and copyright notices, then deleted 
them
from my local system.)

The format described in the technote.txt is relatively simple to implement in
libarchive from scratch.  Anyone who has worked with libarchive even a little 
should
be able to use that technote to implement a reader for the RAR format.

The decompression is trickier.  Because of the license terms, we can't copy any 
of
the unrar source code.  Which means libarchive would have to link against a 
separate
library that implemented the RAR decompression algorithm.  As far as I know, no 
such
library exists.   Without that, there's very little that could be done.

Original comment by kientzle@gmail.com on 10 Oct 2009 at 3:59

• Added labels: Type-Enhancement, Priority-None
• Removed labels: Type-Defect, Priority-Medium

[GoogleCodeExporter]

[7zip][1] has a rar module make it able to extract RAR files. But it's 
certainly not
a library.

The source code is in p7zip_9.04_src_all.tar.bz2/CPP/7zip/Archive/Rar, can this 
code
be reused?

[1]: http://p7zip.sourceforge.net/

Original comment by azu...@gmail.com on 14 Oct 2009 at 8:04

[GoogleCodeExporter]

Original comment by kientzle@gmail.com on 29 Oct 2009 at 3:44

• Added labels: Milestone-Later, Component-libarchive

[GoogleCodeExporter]

p7zip seems to have bundled the (non-free) unrar source code, so it's probably 
not
any help.

Original comment by a.henrik...@gmail.com on 21 Dec 2009 at 12:43

[GoogleCodeExporter]

VLC media player has support for playing files inside RAR archive(s), so maybe
looking at it's code might help.

modules/stream_filter/rar.c from
http://download.videolan.org/pub/videolan/vlc/1.0.4/vlc-1.0.4.tar.bz2

or quick look into the git repository
http://git.videolan.org/?p=vlc.git;a=blob;f=modules/stream_filter/rar.c

Original comment by ondra.pe...@gmail.com on 5 Jan 2010 at 2:45

[GoogleCodeExporter]

The VLC rar.c only handles *uncompressed* files inside a rar file. Might be 
okay for a 
video program but not so useful for libarchive.

Original comment by arne@rfc2549.org on 20 Feb 2010 at 2:21

[GoogleCodeExporter]

The author of the program "The Unarchiver" wrote his own implementation of a 
RAR decompressor from scratch. The relevant code can be found in his hg 
repository in googlecode.
http://code.google.com/p/theunarchiver/

You can browse from 
http://code.google.com/p/theunarchiver/source/browse/XADMaster?r=tip.

The relavant RAR files don't show up in the above link for some reason. Here's 
another link with his code plus some of my own work (for license clarification 
and a C wrapper for his project).
http://gitorious.org/theunarchiver/theunarchiver/trees/master/XADMaster

The relevant code is written from scratch, as he notes in a log message in an 
hg commit. He's stated he's willing to relicense his work under BSD license 
(it's currently LGPL).
http://wakaba.c3.cx/sup/kareha.pl/1151796773/852

I've volunteered to port his implementation to libarchive. His code is written 
in Objective-C and would need some major work to remove parts of the code that 
are only useful for The Unarchiver.

Original comment by mcitadel@gmail.com on 5 Nov 2010 at 7:38

[GoogleCodeExporter]

As long as the code is suitably licensed (ideally using exactly the same 
license as the other core libarchive files), I have no objections to this.

I presume you're planning to adapt the code to plain C rather than Objective-C?

I would suggest you start with the bare format (without the compression) to 
understand the libarchive interfaces.  You should be able to get support for 
uncompressed RAR archives working fairly quickly.  That would also give a 
chance for one of the folks with a lot of libarchive experience (myself, Joerg, 
or Michihiro, for example) to give you some feedback on the project style.  
That's how we got Zip support working:  First by implementing the archive 
format without compression, getting that fully working and tested, then adding 
the compression.  (Fortunately, we could just use zlib for the Zip compression.)

I'm looking forward to seeing your first contribution!

Original comment by kientzle@gmail.com on 7 Nov 2010 at 12:44

[GoogleCodeExporter]

If someone is interested, I made a gvfs backend for uncompressed multipart rar 
files a while ago.

The source patch is here:
http://bugzilla.gnome.org/show_bug.cgi?id=569126

64bit Ubuntu package (should work on 10.04, 10.10) available here:
http://walmis.balticum-tv.lt/share/gvfs-rar_1.6.0-1ubuntu2_amd64.deb
Mount option should appear by right clicking the rar file in nautilus.

Original comment by wal...@gmail.com on 26 Jan 2011 at 3:38

[GoogleCodeExporter]

Here's my first attempt at getting something implemented for RAR support. This 
applies against current SVN.

Some notes follow.

This implementation of RAR support only reads uncompressed files. There is no 
support for compressed files, multivolume RAR files, or files encrypted with a 
password.

This implementation will only read RAR files whose host OS has been set as 
WIN32 or UNIX. In other words, it will only read RAR files created on Windows 
or UNIX machines. This is because of how RAR file headers store the file 
attributes of the host OS. I only have access to Linux and Windows machines and 
thus could only work with file attributes from both machines.

The new C source code file was formatted using 'astyle --style=bsd -c -s2'.

I tested RAR files with directories, regular files, and symlinks. I've tested 
large file support. I implemented support for UTF8 encoded filenames though I 
could not properly test this. Also, I could not properly test parsing arctime 
from the EXTTIME structure in a RAR header.

I am subscribed to libarchive-discuss so feel free to send feedback/comments 
there and I will read them at my leisure.

Original comment by mcitadel@gmail.com on 22 Jun 2011 at 11:00

Attachments:

• rar-implementation.patch

[GoogleCodeExporter]

Here's a new patch to improve RARv3 support, in particular, the parsing of 
sub-blocks (comments).

Original comment by mcitadel@gmail.com on 24 Jun 2011 at 1:04

Attachments:

• rar-implementation.patch

[GoogleCodeExporter]

This does look like a very good start!

Before I commit this, could you please add a basic test:  You can copy 
libarchive/test/test_read_format_ar.c to get started.  This just opens a sample 
file (a RAR file that is UUencoded in your case) and reads the entries and 
verifies basic properties of the entries.  After you get a basic test working, 
it's easy to extend it to verify that you properly read different kinds of RAR 
files (such as files from different host platforms).

I also suggest you add your sample file to the list of files used by 
test_fuzz.c.  The "fuzz test" tries to crash your code by randomly altering a 
sample file.  It has been very helpful in finding common programming errors.

Original comment by kientzle@gmail.com on 26 Jun 2011 at 12:08

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Here is an updated patch adding tests.

Also, there is a fix for a renamed file in the tests directory 
(libarchive/test/test_archive_read_support.c > 
libarchive/test/archive_read_support.c). I included that fix in this patch. I 
can update the patch again once the renaming is fixed.

Original comment by mcitadel@gmail.com on 26 Jun 2011 at 9:05

Attachments:

• rar-implementation.patch

[GoogleCodeExporter]

I've committed your patch.  (I also fixed the renaming issue; thanks for 
pointing that out.)

I've left the RAR support out of archive_read_support_format_all() until the 
test failures are fixed.  Specifically, both test_fuzz and test_read_format_rar 
are consistently crashing here with this message:

libarchive_test(78034) malloc: *** error for object 0x104209250: incorrect 
checksum for freed object - object was probably modified after being freed.

Here's the backtrace that shows where malloc() is detecting the failure; it 
probably happened somewhat earlier than this:
#0  0x00007fff88f5576d in malloc_error_break ()
#1  0x00007fff88f568c4 in szone_error ()
#2  0x00007fff88e82c83 in tiny_free_list_remove_ptr ()
#3  0x00007fff88e801ee in szone_free_definite_size ()
#4  0x00000001000d3267 in archive_string_free (as=0x10482bbf0) at 
/Users/tim/projects/libarchive/trunk/libarchive/archive_string.c:238
#5  0x00000001000d8d46 in archive_mstring_clean (aes=0x10482bbf0) at 
/Users/tim/projects/libarchive/trunk/libarchive/archive_string.c:3654
#6  0x000000010009b693 in archive_entry_clear (entry=0x10482ba00) at 
/Users/tim/projects/libarchive/trunk/libarchive/archive_entry.c:161
#7  0x00000001000a110e in _archive_read_next_header2 (_a=0x10482b200, 
entry=0x10482ba00) at 
/Users/tim/projects/libarchive/trunk/libarchive/archive_read.c:406
#8  0x00000001000a1318 in _archive_read_next_header (_a=0x10482b200, 
entryp=0x7fff5fbfec90) at 
/Users/tim/projects/libarchive/trunk/libarchive/archive_read.c:465
#9  0x00000001000da67a in archive_read_next_header (a=0x10482b200, 
entry=0x7fff5fbfec90) at 
/Users/tim/projects/libarchive/trunk/libarchive/archive_virtual.c:133
#10 0x000000010004a1a1 in test_basic () at 
/Users/tim/projects/libarchive/trunk/libarchive/test/test_read_format_rar.c:77
#11 0x000000010004afbf in test_read_format_rar () at 
/Users/tim/projects/libarchive/trunk/libarchive/test/test_read_format_rar.c:171
#12 0x00000001000072ae in test_run (i=96, tmpdir=0x7fff5fbff360 
"/var/folders/64/64A1HgeYHKW11UT3X70fok+++TI/-Tmp-//libarchive_test.2011-06-26T1
7.43.56-000") at 
/Users/tim/projects/libarchive/trunk/libarchive/test/main.c:2038
#13 0x0000000100008430 in main (argc=3, argv=0x7fff5fbff4a0) at 
/Users/tim/projects/libarchive/trunk/libarchive/test/main.c:2455

Original comment by kientzle@gmail.com on 27 Jun 2011 at 12:47

[GoogleCodeExporter]

Ok. Here's a new patch fixing the above issues. I suppose this was a matter of 
using CFLAGS="-g -O0". I tested this without adding CFLAGS.

I've also added a test for one of the RAR files I forgot to add in the previous 
patch. This lead to finding a problem with test fuzz, where it would fail when 
the size was less than 100. This patch fixes that as well.

Original comment by mcitadel@gmail.com on 27 Jun 2011 at 1:39

Attachments:

• rar-implementation.patch

[GoogleCodeExporter]

I'm seeing this on the tests as well:

Totals:
  Tests run:                1
  Tests failed:             1
  Assertions checked:      90
  Assertions failed:       65
  Skips reported:           0

Failing tests:
  96: test_read_format_rar (65 failures)

Details for failing tests: /tmp/libarchive_test.2011-06-27T10.41.15-000

$ ls -R /tmp/libarchive_test.2011-06-27T10.45.09-000
/tmp/libarchive_test.2011-06-27T10.45.09-000:
test_read_format_rar  test_read_format_rar.log

/tmp/libarchive_test.2011-06-27T10.45.09-000/test_read_format_rar:
test_read_format_rar_noeof.rar  test_read_format_rar.rar  
test_read_format_rar_subblock.rar

Looks like the test doesn't properly #ifdef on RAR support being enabled or 
something?

Original comment by dpmc...@gmail.com on 27 Jun 2011 at 3:47

Attachments:

• test_read_format_rar.log

[GoogleCodeExporter]

I'm not getting any issues running the tests (using the last patch I 
submitted). Could you try cleaning and rebuilding libarchive? Would you mind 
telling me what system your building/running on?

There's no #ifdef or configure switch that disables rar support. It's basically 
an implementation for rar support written from scratch with no external 
dependencies, so I didn't see the need to have the option to disable rar 
support.

Original comment by mcitadel@gmail.com on 27 Jun 2011 at 9:01

[GoogleCodeExporter]

I've committed your latest patch but test_read_format_rar still fails.
all of failures were caused by the test of time stamps. 

Original comment by ggcueroad@gmail.com on 28 Jun 2011 at 5:36

Attachments:

• test_read_format_rar.log

[GoogleCodeExporter]

Looks like the RAR code converts Y/M/D/H/M/S time values into a timestamp using 
mktime(), which is locale-sensitive.  If RAR really stores times in the local 
time zone, you might not be able to do any better; you'll just have to either 
not test the time values returned or do some sloppier testing (test that the 
values returned are within 24 hours of the expected value, for instance).

If RAR times are supposed to be stored in GMT or UTC, you can copy the 
following portable function that I developed for the ISO9660 reader:

static time_t
time_from_tm(struct tm *t)
{
#if HAVE_TIMEGM
        /* Use platform timegm() if available. */
        return (timegm(t));
#elif HAVE__MKGMTIME64
        return (_mkgmtime64(t));
#else
        /* Else use direct calculation using POSIX assumptions. */
        /* First, fix up tm_yday based on the year/month/day. */
        mktime(t);
        /* Then we can compute timegm() from first principles. */
        return (t->tm_sec + t->tm_min * 60 + t->tm_hour * 3600
            + t->tm_yday * 86400 + (t->tm_year - 70) * 31536000
            + ((t->tm_year - 69) / 4) * 86400 -
            ((t->tm_year - 1) / 100) * 86400
            + ((t->tm_year + 299) / 400) * 86400);
#endif
}

Original comment by kientzle@gmail.com on 28 Jun 2011 at 5:56

[GoogleCodeExporter]

When I ran test_read_format_rar on Windows, I got a following error (except 
about time stamps):

..\..\..\libarchive\libarchive\test\test_read_format_rar.c(67): 
"testdir/test.txt" != archive_entry_pathname(ae)
              "testdir/test.txt" = "testdir/test.txt" (length 16)
      archive_entry_pathname(ae) = "testdir\test.txt" (length 16)

That means the RAR reader does not replace  a '\' character by a '/' character 
on
Windows platform.
Other format readers replace it or add '/' as a path separator on every 
platform,
so you should do the replacement on every platform.

Original comment by ggcueroad@gmail.com on 28 Jun 2011 at 7:06

[GoogleCodeExporter]

A new patch to address the latest issues. RAR files are stored with localtime 
I'm afraid, so the test is changed to simply check that {a,c,m}time is set.

Also, I had thought '\' were not converted on Windows.

Original comment by mcitadel@gmail.com on 28 Jun 2011 at 4:56

Attachments:

• rar-implementation.patch

[GoogleCodeExporter]

I've just committed this.  All of the tests seem are now passing for me.  Nice 
work!

Libarchive's interfaces always use '/' for the path separator, regardless of 
platform.

Original comment by kientzle@gmail.com on 29 Jun 2011 at 4:49

[GoogleCodeExporter]

When I made the RAR archive which had files whose name was non ASCII,
I got empty file names.

The result was:
$ bsdtar.exe tvf test_jp2.rar
-rw-r--r--  0 0      0           8 3 19 03:27 (null)
-rw-r--r--  0 0      0           4 3 19 03:28 (null)

I made this patch and applied it.
$ svn diff
Index: libarchive/archive_read_support_format_rar.c
===================================================================
--- libarchive/archive_read_support_format_rar.c        (revision 3447)
+++ libarchive/archive_read_support_format_rar.c        (working copy)
@@ -489,8 +489,8 @@
   {
     if (filename_size != strlen(filename))
     {
-      memcpy(filename, p + strlen(filename), filename_size - strlen(filename));

-      filename[filename_size - strlen(filename)] = '\0';
+      memcpy(filename, p + strlen(filename) +1, filename_size - strlen(filename
) -1);
+      filename[filename_size - strlen(filename) -1] = '\0';
     }
     sconv = archive_string_conversion_from_charset(&a->archive, "UTF-8", 1);
     if (sconv == NULL)

Result after patched:
$ bsdtar.exe tvf test_jp2.rar
-rw-r--r--  0 0      0           8 3 19 03:27 o`"W[.t
bsdtar.exe: Pathname cannot be converted from UTF-8 to current locale.
-rw-r--r--  0 0      0           4 3 19 03:28 (null)
bsdtar.exe: Error exit delayed from previous errors.

I got incorrect file names and some conversion errors.

Following text is the dump data of the RAR file.
$ hexdump -C test_jp2.rar
00000000  52 61 72 21 1a 07 00 cf  90 73 00 00 0d 00 00 00  |Rar!.....s......|
00000010  00 00 00 00 a4 b6 74 20  92 38 00 08 00 00 00 08  |......t .8......|
00000020  00 00 00 02 3f 9b 1b c3  7a 1b 73 3e 14 30 13 00  |....?...z.s>.0..|
00000030  20 00 00 00 8a bf 8e 9a  2e 74 78 74 00 6f 60 22  | ........txt.o`"|
00000040  57 5b 2e 74 00 78 74 00  f0 f4 0f 51 8a bf 8e 9a  |W[.t.xt....Q....|
00000050  82 a9 82 c8 52 98 74 20  92 34 00 04 00 00 00 04  |....R.t .4......|
00000060  00 00 00 02 18 f8 c8 44  86 1b 73 3e 14 30 0f 00  |.......D..s>.0..|
00000070  20 00 00 00 95 5c 2e 74  78 74 00 88 40 68 2e 74  | ....\.txt..@h.t|
00000080  78 00 74 00 b0 a0 25 26  6c 69 73 74 c4 3d 7b 00  |x.t...%&list.={.|
00000090  40 07 00                                          |@..|
00000093

the byte sequence of a first file "漢字.txt":
8a bf 8e 9a  2e 74 78 74 00 6f 60 22 57 5b 2e 74 00 78 74   (19 bytes)

the byte sequence of a second file "表.txt":
95 5c 2e 74  78 74 00 88 40 68 2e 74 78 00 74 (15 bytes)

Those file names seems to be split by zero, and its first half is
a file name in natural character-set(in this case it is CP932), and
its second half is a unicode file name, but it does not seems that
the unicode file name is a natural unicode, not UTF-8 nor UTF-16!

I compared several forms of that files.   
"漢字.txt":
UTF-8 form     : e6 bc a2 e5 ad 97 2e 74  78 74
UTF-16BE form  : 6f 22 5b 57 00 2e 00 74  00 78 00 74
the second half: 6f 60 22 57 5b 2e 74 00 78 74

"表.txt":
UTF-8 form     : e8 a1 a8 2e 74 78 74
UTF-16BE form  : 88 68 00 2e 00 74 00 78  00 74
the second half: 88 40 68 2e 74 78 00 74

So I think those were converted from its UTF-16 string in some rules
so as to trim 0x00 from the UTF-16 string as much as they can.

If my guess is right, the handling of file names should be:

    if (filename_size != (fn_bytes = strlen(filename)))
    {
      memcpy(xxfilename, p + fn_bytes +1, filename_size - fn_bytes -1);
      /* decode rar's unicode. */
      fn_bytes = decode_rarunicode_to_utf16be(filename, xxfilename, filename_size - fn_bytes -1);
      sconv = archive_string_conversion_from_charset(&a->archive, "UTF-16BE", 1);
    } else
      sconv = archive_string_conversion_from_charset(&a->archive, "UTF-8", 1);
  .......  
  .......  
  if (archive_entry_copy_pathname_l(entry, filename, fn_bytes, sconv))
  .......  

Original comment by ggcueroad@gmail.com on 30 Jun 2011 at 8:42

Attachments:

• test_jp2.rar

[GoogleCodeExporter]

This attached RAR file, which was broken by test_fuzz, caused a infinity loop 
on at least both FreeBSD and Cygwin.

(gdb) run tf inf.rar
Starting program: /usr/proj/libarchive/cab/bsdtar tf inf.rar
test.txt
testlink
testdir/test.txt
\314cw\332>dw^C
Program received signal SIGINT, Interrupt.
0x285e475f in read () from /lib/libc.so.7
(gdb) bt
#0  0x285e475f in read () from /lib/libc.so.7
#1  0x285e4596 in lseek () from /lib/libc.so.7
#2  0x08060150 in file_skip_lseek (a=0x28725400, client_data=0x2871d060, 
    request=1073741824) at libarchive/archive_read_open_filename.c:288
#3  0x08060283 in file_skip (a=0x28725400, client_data=0x2871d060, 
    request=1073741824) at libarchive/archive_read_open_filename.c:322
#4  0x0805896a in client_skip_proxy (self=0x28704060, 
request=8390552989898637251)
    at libarchive/archive_read.c:195
#5  0x0805a5f2 in advance_file_pointer (filter=0x28704060, 
    request=8391162079611977667) at libarchive/archive_read.c:1228
#6  0x0805a378 in __archive_read_filter_consume (filter=0x28704060, 
    request=8391162079611977728) at libarchive/archive_read.c:1175
#7  0x0805a321 in __archive_read_consume (a=0x28725400, 
request=8391162079611977728)
    at libarchive/archive_read.c:1163
#8  0x0807cb20 in archive_read_format_rar_read_data_skip (a=0x28725400)
    at libarchive/archive_read_support_format_rar.c:377
#9  0x08059605 in archive_read_data_skip (_a=0x28725400)
    at libarchive/archive_read.c:651
#10 0x0805004a in read_archive (bsdtar=0xbfbfe4c4, mode=116 't', writer=0x0)
    at tar/read.c:299
#11 0x0804f8d1 in tar_mode_t (bsdtar=0xbfbfe4c4) at tar/read.c:84
#12 0x0804cdbe in main (argc=3, argv=0xbfbfe5e4) at tar/bsdtar.c:658
(gdb) q

Original comment by ggcueroad@gmail.com on 30 Jun 2011 at 10:36

Attachments:

• inf.rar

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Would you mind if I added the test_jp2.rar file to the test suite? I haven't 
been able to generate a rar file with unicode file names.

Original comment by mcitadel@gmail.com on 30 Jun 2011 at 11:01

[GoogleCodeExporter]

Of course not.

Original comment by ggcueroad@gmail.com on 30 Jun 2011 at 3:23

[GoogleCodeExporter]

The unicode file names are encoded in the rar file in some form. Could you 
provide another rar with longer unicode file names, to include some inside a 
unicode directory?

Original comment by mcitadel@gmail.com on 30 Jun 2011 at 4:06

[GoogleCodeExporter]

Here is an additional sample file which has two files in Russian.
test_ru.rar can be tested on Windows platform in variety of countries
more than test_jp2.rar can, but I'd like you to use both files. 

First file "ПРИВЕТ":
UTF-8 form   : d0 9f d0 a0 d0 98 d0 92 d0 95 d0 a2
UTF-16BE form: 04 1f 04 20 04 18 04 12 04 15 04 22

Second file "привет":
UTF-8 form   : d0 bf d1 80 d0 b8 d0 b2 d0 b5 d1 82
UTF-16BE form: 04 3f 04 40 04 38 04 32 04 35 04 42

And you can copy test_read_format_cpio_filename.c and modify it for tests for
the unicode file names.

Original comment by ggcueroad@gmail.com on 30 Jun 2011 at 4:29

Attachments:

• test_ru.rar

[GoogleCodeExporter]

Here is a new sample file which has long file names.

The file includes two files :
表だよ\新しいフォルダ\新規テキスト ドキュメント.txt
表だよ\漢字長いファイル名long-filename-in-漢字.txt

Original comment by ggcueroad@gmail.com on 1 Jul 2011 at 4:26

Attachments:

• longname_jp.rar

[GoogleCodeExporter]

I decided to work on the compression first before tackling the unicode support 
or test_fuzz issues. This patch will implement read support for RAR's lzss 
compression (normal, fast, and fastest compression).

I've been using theunarchiver to help with getting the lzss compression 
working. The author wrote the RAR implementation from scratch. There isn't much 
code reused from theunarchiver in use in libarchive, primarily because 
theunarchiver is written in Objective-C and is not a library. In any case, I 
have received permission to use code from theunarchiver and relicense code for 
the RAR implementation in theunarchiver in BSD. See 
http://wakaba.c3.cx/sup/kareha.pl/1151796773/852.

This implementation took some time, and I would like to submit what I have so 
far. I believe there is still much room for improvement. Even so, this 
implementation is already surpassing the official "unrar" program in terms of 
performance. The two log files show basic stats for time runs on bsdtar and 
unrar on my system. bsdtar was built using default optimizations and unrar is 
from whatever unrar package that is available in Debian.

Original comment by mcitadel@gmail.com on 15 Jul 2011 at 1:42

Attachments:

• rar-lzss.patch
• bsdtar_extract.log
• unrar_extract.log

[GoogleCodeExporter]

BTW, I am interested in getting commit access solely for the purpose of working 
on the RAR implementation for libarchive.

Original comment by mcitadel@gmail.com on 15 Jul 2011 at 1:43

[GoogleCodeExporter]

Nice work!

I've committed your patch.

Contact me directly at kientzle@freebsd.org to discuss getting commit access.

Original comment by kientzle@gmail.com on 16 Jul 2011 at 6:55

[GoogleCodeExporter]

test_fuzz often stalls at reading RAR files at least on both Linux and FreeBSD.

Andres, You should execute a following script for one or two hours in order
to eliminate these sort of test_fuzz failure caused by the RAR reader.
#!/bin/sh
while true
do
  echo -n "."
  ./libarchive_test test_fuzz > /dev/null
  sleep 1
done

Original comment by ggcueroad@gmail.com on 27 Jul 2011 at 7:54

[GoogleCodeExporter]

I'll handle this one.

Original comment by mcitadel@gmail.com on 13 Aug 2011 at 6:55

• Changed state: Accepted

[GoogleCodeExporter]

Original comment by mcitadel@gmail.com on 13 Aug 2011 at 6:57

[GoogleCodeExporter]

There are some memory access error when reading a broken file.
Please see an attached log file.

Original comment by ggcueroad@gmail.com on 15 Aug 2011 at 2:30

Attachments:

• rar-mem.log

[GoogleCodeExporter]

Original comment by amejia004@gmail.com on 16 Jan 2012 at 4:18

[GoogleCodeExporter]

How is the rar support now ? Can libarchive support compressed rar files ?

Original comment by renenglish on 4 Jun 2014 at 11:20

[GoogleCodeExporter]

RAR is supported since libarchive 3.0:
https://github.com/libarchive/libarchive/wiki/ReleaseNotes#libarchive-302

multi-volume RAR is apparently planned for the 3.1.x release.

Original comment by hadessuk@googlemail.com on 29 Aug 2014 at 9:32