rikyoz / bit7z

A C++ static library offering a clean and simple interface to the 7-zip shared libraries.
https://rikyoz.github.io/bit7z
Mozilla Public License 2.0
623 stars 113 forks source link

Is it possible that i can steam the volume rather than 2 step? As I research 7zip.exe doing it using steam which will not produce extra combination zip file. #90

Closed jf3096 closed 11 months ago

jf3096 commented 2 years ago

bit7z version

3.1.x

7-zip version

v21.07

7-zip DLL used

7z.dll

MSVC version

2017

Architecture

x86_64

Which version of Windows are you using?

Windows 10

Bug description

here is my code:

void extractVolumeExample()
{
    Bit7zLibrary lib{ L"C:/Users/admin/source/repos/bit7z-demo/libs/bit7z/dll/7z.dll" };
    BitExtractor extractor{ lib, BitFormat::Split };

    // step 1: after extract, it combine all the volumes zip file into one
    extractor.extract(L"D:/test/volumes/Anaconda3-2022.05-Windows-x86_64.7z.001", L"D:/test/extract-to");
    // step 2: then what i need to do is to unzip it.
    extractFileExample(L"D:/test/extract-to/Anaconda3-2022.05-Windows-x86_64.7z", L"D:/test/extract-to/");

    // Question: is it possible that i can steam the volume rather than 2 step? As I research 7zip.exe doing it using steam which will not produce extra combination zip file
    std::cout << "Done";
}

Is it possible that i can steam the volume rather than 2 step? As I research 7zip.exe doing it using steam which will not produce extra combination zip file.

Steps to reproduce

No response

Expected behavior

No response

Relevant compilation output

No response

Code of Conduct

rikyoz commented 2 years ago

Hi!

Is it possible that i can steam the volume rather than 2 step?

There's no direct way to achieve this in bit7z, but you might be able to accomplish it by using a custom class implementing std::iostream and using a memory buffer. This class would implement both std::ostream and std::istream; hence you would be able to use both:

In this way, you might be able to avoid extracting twice to the filesystem. However, I never tested this approach, so I'm not entirely sure it would work in practice.

Alternatively, if the input archive is not too big, you might extract all of it to a memory buffer and then extract this latter using BitMemExtractor::extract. However, since the input archive is split, it's probably too big for this last approach.

As I research 7zip.exe doing it using steam which will not produce extra combination zip file.

I don't know how 7-zip works in this specific case; however, in other situations, it often uses temporary files, so it might be that it actually implements a two-step extraction to the filesystem under the hood.

jf3096 commented 2 years ago

I don't know how 7-zip works in this specific case; however, in other situations, it often uses temporary files Thank you for your reply. I did some test and found out:

  1. No temp file generated. As I did try to look for it in window temp folder and anywhere I can think of
  2. Since 7zip.exe gui allow to pause during unzipping. It first generated the unzip file then it fill it up with more data (I open the unzipped file and monitor it using winhex.exe). It feels like streaming.
rikyoz commented 2 years ago

I don't know how 7-zip works in this specific case; however, in other situations, it often uses temporary files Thank you for your reply. I did some test and found out:

1. No temp file generated. As I did try to look for it in window temp folder and anywhere I can think of

2. Since 7zip.exe gui allow to pause during unzipping. It first generated the unzip file then it fill it up with more data (I open the unzipped file and monitor it using winhex.exe). It feels like streaming.

Ok, interesting! I'll need to investigate further in the 7-zip source code, hoping there's an API it provides for directly extracting split archives. Unfortunately, the 7-zip code is poorly documented.

rikyoz commented 11 months ago

Implemented in v3.2.0 and v4.0.0.