Zygo / bees

Best-Effort Extent-Same, a btrfs dedupe agent
GNU General Public License v3.0
661 stars 55 forks source link

bees isn't deduping any of the files #200

Open bdnbje opened 2 years ago

bdnbje commented 2 years ago

I have been running beesd by following this article. I have it running for 2 days and i still cant see any results. It's a freshly built btrfs drive for testing before i deploy it with my data. pi@BedroomPi:~ $ sudo btrfs filesystem df torrent Data, single: total=7.00GiB, used=6.71GiB System, DUP: total=8.00MiB, used=64.00KiB Metadata, DUP: total=1.00GiB, used=20.00MiB GlobalReserve, single: total=17.62MiB, used=0.00B pi@BedroomPi:~ $ sudo systemctl status beesd@cefc2d3e-f3bd-4d05-82e0-ee6256944e6 a.service ● beesd @cefc2d3e-f3bd-4d05-82e0-ee6256944e6a.service - Bees (cefc2d3e-f3bd-4d05- Loaded: loaded (/lib/systemd/system/beesd@.service; enabled; vendor preset: e Active: active (running) since Mon 2021-11-01 20:56:44 IST; 6h ago Docs: https://github.com/Zygo/bees Main PID: 1049 (bees) Tasks: 11 (limit: 1831) CPU: 30.741s CGroup: /system.slice/system-beesd.slice/beesd @cefc2d3e-f3bd-4d05-82e0-ee6256 └─1049 /usr/lib/bees/bees --no-timestamps /run/bees/mnt/cefc2d3e-f3bd Nov 02 02:57:26 BedroomPi beesd[1049]: hash_prefetch[1081]: hash_bump=0. Nov 02 02:57:26 BedroomPi beesd[1049]: hash_prefetch[1081]: inserted_blo Nov 02 02:57:26 BedroomPi beesd[1049]: hash_prefetch[1081]: matched_0=0. Nov 02 02:57:26 BedroomPi beesd[1049]: hash_prefetch[1081]: open_clear=0 Nov 02 02:57:26 BedroomPi beesd[1049]: hash_prefetch[1081]: readahead_ms Nov 02 02:57:26 BedroomPi beesd[1049]: hash_prefetch[1081]: root_clear=0 Nov 02 02:57:26 BedroomPi beesd[1049]: hash_prefetch[1081]: scan_block=0 Nov 02 02:57:26 BedroomPi beesd[1049]: hash_prefetch[1081]: scanf_extent Nov 02 03:11:44 BedroomPi beesd[1049]: crawl_writeback[1083]: Saving crawl state Nov 02 03:11:44 BedroomPi beesd[1049]: crawl_writeback[1083]: Nothing to ### `save I have three files 2 are the same file and the other is the same file with some changes.

Zygo commented 2 years ago

We're going to need to see more log data. All you have pasted there says that bees isn't doing anything, and we already know that. Start from the beginning up to the first error.

Is this Pi running in 32-bit or 64-bit mode? #186 may still be affecting 64-bit Raspberry PI (arm64) systems.

Zygo commented 2 years ago

If you get an inode with ls -li, can you look up its inode with btrfs ins ino <inode number from ls> . on the 64-bit kernel? If not, you may have found a kernel bug.

I don't see any other way to hit the EFAULT case in the kernel code, but I also don't see why copy_to_user would fail, unless it's 32-bit userspace and 64-bit kernel or something?

bdnbje commented 2 years ago

this is what i get from ls -li pi@BedroomPi:~/torrent $ ls -li total 5975292 262 -rwxrw-rw- 1 pi pi 2147483648 Oct 27 20:06 'xx' 263 -rwxrw-rw- 1 pi pi 2147483648 Nov 1 17:37 'xx2' 261 -rwxrw-rw- 1 pi pi 2147483648 Oct 27 20:20 'xx3' pi@BedroomPi:~/torrent $ ^C pi@BedroomPi:~/torrent $ btrfs ins ino 263 btrfs inspect-internal inode-resolve: exactly 2 arguments expected, 1 given pi@BedroomPi:~/torrent $ btrfs ins ino 261 btrfs inspect-internal inode-resolve: exactly 2 arguments expected, 1 given pi@BedroomPi:~/torrent $ btrfs ins ino 262 btrfs inspect-internal inode-resolve: exactly 2 arguments expected, 1 given

full logs

and yes i am running it in 64bit mode

bdnbje commented 2 years ago

checking the full log the only error i could find is Nov 01 18:11:49 BedroomPi beesd[408]: crawl_5[593]: exception (ignored): exception type std::runtime_error: FIXME: bailing out here, need to fix this further up the call stack

Zygo commented 5 months ago

Commit 3839690 probably fixes this issue too. Please test.