search

markfasheh / duperemove

Tools for deduping file systems
GNU General Public License v2.0
795 stars 80 forks source link

Dead loop when iterating fiemap #211

Closed cuihaoleo closed 5 years ago

cuihaoleo commented 5 years ago

OS: Arch Linux with 4.19.12-arch1-1-ARCH kernel Latest code from git (13c29f867e5e20e088620161a7114a7751597b06).

I saw that duperemove was stuck on a spefific file on my btrfs. It was consuming 100% CPU.

I compiled a debug version. The following debug message is from duperemove -hv --cpu-threads=1 --io-threads=1 --debug /data/android/Sdk/extras/android/gapid/3/linux/x86_64/gapis 2>&1 | head -n 2000: debug.txt

It is obviously there is a dead loop when iterating fiemap.

Hi-Angel commented 5 years ago

It was consuming 100% CPU.

Do you also see a slow ongoing increase in memory usage by duperemove?

I'm seeing a similar problem with 4.20, and trying to figure whether mine is a new issue.

Hi-Angel commented 5 years ago

Do you also see a slow ongoing increase in memory usage by duperemove?

I'm seeing a similar problem with 4.20, and trying to figure whether mine is a new issue.

Okay, so, with linux 4.20.3, and latest as of writing the words duperemove from git I do not experience that anymore.