search

weltkante / managed-lzma

C# implementation of LZMA and 7zip
MIT License
140 stars 22 forks source link

Multithreaded operation possilble? #29

Closed danwalmsley closed 5 years ago

danwalmsley commented 5 years ago

I would be happy to implement multithreaded extraction if you can point me in the right direction and perhaps there is a reason its not possible?

Dan

weltkante commented 5 years ago

The underlying engine has multithreading built in, you just configure how many threads you want to use. It might be possible that I didn't expose that setting to the user API though. The properties you have to look at are here

danwalmsley commented 5 years ago

Hi, thanks for your rapid response its greatly appreciated. It looks as though that only applies to compressing stuff? how about decompressing?

danwalmsley commented 5 years ago

btw this is the code im using to unpack a 7z file... https://github.com/VitalElement/AvalonStudio/blob/azure-packages/AvalonStudio/PackageManager/PackageManager.cs#L297

danwalmsley commented 5 years ago

I sent you a PR to fix an issue I had extracting some archives... https://github.com/weltkante/managed-lzma/pull/30

weltkante commented 5 years ago

Decompressing cannot be made multithreaded if you use the default settings, because in the default settings during compression all files will be concatenated into a single stream. Decompressing a single stream is a sequential operation so there is no speedup in using multiple threads.

If you are only decompressing your own archives (i.e. if you control compression) you could generate multiple compressed streams instead of a single stream, that would allow to use multithreaded decompression. Compression ratio would depend a lot on the right distribution of files among the streams though, if you do a bad job it would reduce compression ratio a lot.

I'll look at the bugfix later today when I'm home.

danwalmsley commented 5 years ago

I understand, many thanks for taking your time to help me. I will close this issue now. All the best