Ne-Lexa / php-zip

PhpZip is a php-library for extended work with ZIP-archives.
MIT License
493 stars 60 forks source link

Adding files to large zip #59

Closed fakhamatia closed 4 years ago

fakhamatia commented 4 years ago

I use this way to zip part of a directory to a zip file But I have a problem, When zip file size increases time adding files to zip also increases and when size is large adding files is very slow Is this way is very bad?

        $zipFile = new \PhpZip\ZipFile();
        $zipFile->setCompressionLevel(\PhpZip\Constants\ZipCompressionLevel::SUPER_FAST);
        $zipFile->openFile("temp/file.zip");
        $zipFile->addFile($file, null, \PhpZip\Constants\ZipCompressionMethod::STORED);
        $zipFile->saveAsFile("temp/file.zip");
        $zipFile->close();
Ne-Lexa commented 4 years ago

Hello. Delete the line

$zipFile->setCompressionLevel(\PhpZip\Constants\ZipCompressionLevel::SUPER_FAST);

You change the compression level and therefore the decompression of the content and its compression by the SUPER_FAST compression method is called.

fakhamatia commented 4 years ago

I delete that line, not different I want to show user progress of zipping If I use addDirRecursive its zip files in all at once in an unknown time I use iterator and get the list of files in a directory and with subdirectories I divide the list by 100 steps, in step 0 I create empty file saveAsFile, and in step 1 openFile and adding one by one file in a for loop with addFile and when for loop end saveAsFile and close, I show 1% to the user and openFile ... until step 100

If I use addDirRecursive files zip in seconds, but in steps way zip in minutes. The closer it gets to the end and the larger the size, the longer the zipping time.

Ne-Lexa commented 4 years ago

I divide the list by 100 steps, in step 0 I create empty file saveAsFile, and in step 1 openFile and adding one by one file in a for loop with addFile and when for loop end saveAsFile and close, I show 1% to the user and openFile ... until step 100

If I use addDirRecursive files zip in seconds, but in steps way zip in minutes.

You are doing something wrong. Each time you recreate the archive and therefore it lasts a very long time. If you need to implement a progress bar, override ZipWriter.

Example:

<?php

use PhpZip\Constants\ZipCompressionMethod;
use PhpZip\IO\ZipWriter;
use PhpZip\Model\ZipContainer;
use PhpZip\ZipFile;

require __DIR__ . '/vendor/autoload.php';

interface ProgressListener
{
    public function onProgress(int $progress);
}

class ZipProgressWriter extends ZipWriter
{
    /** @var ProgressListener|null */
    private $listener;

    public function __construct(ZipContainer $container, ?ProgressListener $listener = null)
    {
        parent::__construct($container);
        $this->listener = $listener;
    }

    public function write($outStream)
    {
        $this->fireProgress(0);
        parent::write($outStream);
        $this->fireProgress(100);
    }

    private function fireProgress(int $progress): void
    {
        if ($this->listener !== null) {
            $this->listener->onProgress($progress);
        }
    }

    protected function writeCentralDirectoryBlock($outStream)
    {
        parent::writeCentralDirectoryBlock($outStream);
        $this->fireProgress(98);
    }

    protected function writeLocalBlock($outStream)
    {
        $zipEntries = $this->zipContainer->getEntries();
        $count = count($zipEntries);

        $progressLimit = 95;

        $i = 0;
        foreach ($zipEntries as $zipEntry) {
            $this->writeLocalHeader($outStream, $zipEntry);
            $this->writeData($outStream, $zipEntry);

            if ($zipEntry->isDataDescriptorEnabled()) {
                $this->writeDataDescriptor($outStream, $zipEntry);
            }

            $progress = (int)floor($i / $count * $progressLimit);
            $this->fireProgress($progress);

            $i++;
        }

        $this->fireProgress($progressLimit);
    }
}

class ZipProgressFile extends ZipFile
{

    /** @var ProgressListener|null */
    private $progressListener;

    /**
     * @param ProgressListener $progressListener
     */
    public function setProgressListener(ProgressListener $progressListener): void
    {
        $this->progressListener = $progressListener;
    }

    public function removeProgressListener(): void
    {
        $this->progressListener = null;
    }

    protected function createZipWriter()
    {
        return new ZipProgressWriter($this->zipContainer, $this->progressListener);
    }
}

$outputFile = sys_get_temp_dir() . '/progress_test.zip';

$progressListener = new class implements ProgressListener {

    public function onProgress(int $progress)
    {
        fwrite(STDOUT, "\r\e[0KProgress \e[1;32m" . $progress . "%\e[m");
    }
};

$zipFile = new ZipProgressFile();
$zipFile->setProgressListener($progressListener);
for ($fileNo = 0; $fileNo < 200; $fileNo++) {
    $entryName = 'file_' . $fileNo . '.txt';
    $contents = random_bytes(1024000);
    $zipFile->addFromString($entryName, $contents, ZipCompressionMethod::DEFLATED);
}
$zipFile->saveAsFile($outputFile);
$zipFile->close();

fwrite(STDOUT, "\n");
Ne-Lexa commented 4 years ago

@fakhamatia, did this solution help you?

fakhamatia commented 4 years ago

Sorry for the late reply. I'm a beginner and I didn't get it right. Why should I add files from the string? and why $contents its random bytes? My files 99.9% JPEG in nested folders and 0.1% HTML Compression time is not worth the reduced size. Can I use STORED? Of course, writing my program is not your job. Thank you for taking the time.

Ne-Lexa commented 4 years ago

Why should I add files from the string? and why $contents its random bytes? My files 99.9% JPEG in nested folders and 0.1% HTML Compression time is not worth the reduced size. Can I use STORED?

This was just an example. You can add images as STORED.