search

knorrie / python-btrfs

Python Btrfs module
GNU Lesser General Public License v3.0
112 stars 22 forks source link

Scrub #23

Open GrahamCobb opened 4 years ago

GrahamCobb commented 4 years ago

Any chance you would consider adding the scrub ioctls to the library?

(When the kernel 5 scrub cancel problem is fixed...) I would love to be able to rewrite my btrfs-scrub-slowly script (part of btrfs-balance-slowly) in python.

Rewriting balance-slowly in python using your library has improved it significantly. Although scrub-slowly is simpler, the timers are hell and it would be great to be able to get access in the script to things like the progress information as I am thinking about adjusting delays if things are running particularly slowly (implying a lot else is going on).

Just hoping you might consider adding it to your ever-growing list :-)

knorrie commented 4 years ago

Sure, sounds like fun! Almost all of my spare time in 2019 sunk into some IRL project, and coincidentally, now is around the time that I start seeing things like this back in my todo list. I was halfway some other feature, to quickly do metadata searches, and after that, this is a good idea.

Question on the side: why are you actually doing btrfs balance? I really haven't been using it that much since I sorted out the -o ssd stuff. All the problems with overallocations and free space fragmentation during normal use are gone.

knorrie commented 4 years ago

Oh, haha, now I see that you're also the submitter of #22

I missed that, and was like "oh, I can tell that other person to use btrfs-balance-slowly is he really wants to do metadata balance". :-D

GrahamCobb commented 4 years ago

Thanks for considering adding it to your list. I look forward to rewriting my scrub-slowly script in Python later in the year. It can't end up worse than the current btrfs-progs scrub (I have now had two goes at fixing problems in that and I think the code is unnecessarily obscure).

knorrie commented 4 years ago

The v12 release is about to happen. After that, this one is in the picture!

GrahamCobb commented 4 years ago

Great! I'm looking forward to trying it out.

Balance has become less of a problem now but scrubs are important and now that I have more and more data on btrfs volumes I find they are not just taking a long time but really disrupting the server due to their CPU and IO loads. I want to play with different ideas about reducing that impact including maybe some adaptive scheduling.