mhx / dwarfs

A fast high compression read-only file system for Linux, Windows and macOS
GNU General Public License v3.0
2.16k stars 58 forks source link

exception thrown in worker thread: class dwarfs::runtime_error: lzma_stream_encoder #224

Closed mindfocus closed 5 months ago

mindfocus commented 5 months ago

use bat in windows 11

@echo off
@chcp 65001
echo 开始处理...
@setlocal
@set "inputFloder=%~1"
@set "outputFile=%~1.dwarfs"
E:\dwarfs-0.9.9-Windows-AMD64-ricepp\bin\mkdwarfs.exe -l 9 -i "%inputFloder%" -o "%outputFile%" --log-level trace
@endlocal
echo 处理结束.
@pause

T 01:29:12.376901 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 462,336 from chunkable offset 4,989,791 T 01:29:12.377074 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 462,592 from chunkable offset 4,990,047 T 01:29:12.377270 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 462,848 from chunkable offset 4,990,303 T 01:29:12.377443 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 463,104 from chunkable offset 4,990,559 T 01:29:12.377616 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 463,360 from chunkable offset 4,990,815 T 01:29:12.377790 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 463,616 from chunkable offset 4,991,071 T 01:29:12.377965 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 463,872 from chunkable offset 4,991,327 T 01:29:12.378167 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 464,128 from chunkable offset 4,991,583 T 01:29:12.378516 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 464,384 from chunkable offset 4,991,839 T 01:29:12.378632 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 464,640 from chunkable offset 4,992,095 T 01:29:12.378712 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 464,896 from chunkable offset 4,992,351 T 01:29:12.378823 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 465,152 from chunkable offset 4,992,607 T 01:29:12.379012 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 465,408 from chunkable offset 4,992,863 T 01:29:12.379109 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 465,664 from chunkable offset 4,993,119 T 01:29:12.379198 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 465,920 from chunkable offset 4,993,375 T 01:29:12.379303 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 466,176 from chunkable offset 4,993,631 T 01:29:12.379379 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 466,432 from chunkable offset 4,993,887 T 01:29:12.379461 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 466,688 from chunkable offset 4,994,143 T 01:29:12.379548 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 466,944 from chunkable offset 4,994,399 T 01:29:12.379666 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 467,200 from chunkable offset 4,994,655 T 01:29:12.379837 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 467,456 from chunkable offset 4,994,911 T 01:29:12.380011 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 467,712 from chunkable offset 4,995,167 T 01:29:12.380185 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 467,968 from chunkable offset 4,995,423 T 01:29:12.380374 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 468,224 from chunkable offset 4,995,679 T 01:29:12.380500 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 468,480 from chunkable offset 4,995,935 T 01:29:12.380593 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 468,736 from chunkable offset 4,996,191 T 01:29:12.380678 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 468,992 from chunkable offset 4,996,447 T 01:29:12.380761 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 469,248 from chunkable offset 4,996,703 T 01:29:12.380842 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 469,504 from chunkable offset 4,996,959 T 01:29:12.380961 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 469,760 from chunkable offset 4,997,215 T 01:29:12.381135 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 470,016 from chunkable offset 4,997,471 T 01:29:12.381314 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 470,272 from chunkable offset 4,997,727 _F 01:29:12.381179 [worker_group.cpp:284] exception thrown in worker thread: class dwarfs::runtime_error: lzma_streamencoder [lzma.cpp:152] T 01:29:12.381453 [segmenter.cpp:1057] appending 256 bytes to block 1 @ 470,528 from chunkable offset 4,997,983 ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯ writing: \?\E:\BaiduYunDownload\06.新东方国学\诗经\诗经.12.黍离.mp3 5 dirs, 0/0 soft/hard links, 186/186 files, 0 other original size: 1.242 GiB, hashed: 367.2 MiB (46 files, 1.191 MiB/s) scanned: 1.062 GiB (163 files, 781 KiB/s), categorizing: 0 B/s saved by deduplication: 183.6 MiB (23 files), saved by segmenting: 90.52 KiB filesystem: 64.21 MiB in 1 blocks (8 chunks, 6/163 fragments, 163 inodes) compressed filesystem: 0 blocks/0 B written ███▍ ▏ 3% 🌕 \?\E:\BaiduYunDownload\06.新东方国学\诗经\诗经.12.黍离.mp3 █▉ ▏1.332 MiB/s [compressing] 0 B/s

mhx commented 5 months ago

Hi and thanks for the report!

You've found a bug that's been in the code for a really long time. The tl;dr is that LZMA encoder objects can be cloned and clones would internally still point to data from the original instance. Depending on whether or not the original instance is freed, if a new instance gets allocated at the same address, or how quickly the memory allocator reuses the freed memory, this would cause lzma_stream_encoder to be called with invalid options.

Fortunately, it's an easy fix. I'll prepare a new release ASAP.

mhx commented 5 months ago

Fixed in v0.9.10.