Out of Memory: compress millions of files piecewise

juninho12 / freearc

Automatically exported from code.google.com/p/freearc

1 stars 0 forks source link

Out of Memory: compress millions of files piecewise #287

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago

What steps will reproduce the problem?
1. Add a large folder (4 million files and 3 TB in this case)
2.
3.

What is the expected output? What do you see instead?
I expected it to archive all the files, saw the out of memory error message 
instead.

What version of the product are you using? On what operating system?
0.666. Windows Server 2003 Standard 32-bit SP2

Please provide any additional information below.
It seems to finish counting the files okay, then CPU usage goes to 100% and 
after a few minutes it gives the out of memory error message.
System has 2GB of RAM. FreeArc.exe was using about 1.6GB and kept it there.

Original issue reported on code.google.com by Nathanie...@gmail.com on 19 Jan 2012 at 6:13

GoogleCodeExporter commented 9 years ago

It requires 50--1000 byte for every file compressed, so no wonder. FreeArc is 
32-bit program (there is no Win64 compiler) so it cannot be fixed

Original comment by bulat.zi...@gmail.com on 19 Jan 2012 at 6:32

Changed state: WontFix

GoogleCodeExporter commented 9 years ago

Does it have to keep it all in memory? Can it finish an archive piecewise?

Original comment by Nathanie...@gmail.com on 19 Jan 2012 at 6:39

GoogleCodeExporter commented 9 years ago

you are right - it had the idea to perform compression piecewise, although 
don't implermented it yet. so i will rename the issue and accept it

Original comment by bulat.zi...@gmail.com on 19 Jan 2012 at 8:09

Changed title: Out of Memory: compress millions of files piecewise
Changed state: Accepted
Added labels: Affects-Usability, Component-Archive, Priority-Low, Type-Enhancement
Removed labels: Priority-Medium, Type-Defect

GoogleCodeExporter commented 9 years ago

when i open 1G file,it report out of Out of Memory
version:0.67 

can you repair it

Original comment by w123...@gmail.com on 5 Mar 2012 at 12:08

GoogleCodeExporter commented 9 years ago

w123dog, i can do it if you will send me the archive. you can try it yourself 
by installing freearc on win64 computer and extracting archive there. also you 
can try with console arc.exe and unarc.exe - any may be better than other for 
this problem. try in cmdline:

arc x archive.arc
unarc x archive.arc

Original comment by bulat.zi...@gmail.com on 5 Mar 2012 at 6:16