auanasgheps / snapraid-aio-script

The definitive all-in-one SnapRAID script on Linux. Diff, sync, scrub are things of the past. Manage SnapRAID and much, much more!
GNU General Public License v3.0
241 stars 36 forks source link

fix get_counts() not parsing large diffs #114

Closed desertwitch closed 2 months ago

desertwitch commented 3 months ago

Difference count numbers longer than 8 digits are not padded with spaces (due to %8u formatter): https://github.com/amadvance/snapraid/blob/c2664c6521aa7df7549a33b4a484427b2ab5f6de/cmdline/scan.c#L1983-L1989

The current regular expression however demands at least one space in front of any difference count number. This causes get_counts() to fail if a single difference count number is longer than 8 digits.

The proposed change with a simplified regular expression does not require a space in front of the number. It accepts such padding spaces, but does not require them anymore. It does require a number, which always is present.

Here is an example script to highlight the issue:

#!/bin/bash
set -o xtrace
echo "  327791 equal" | grep -w '^ \{1,\}[0-9]* equal' | sed 's/^ *//g' | cut -d ' ' -f1
echo "12327791 equal" | grep -w '^ \{1,\}[0-9]* equal' | sed 's/^ *//g' | cut -d ' ' -f1

echo "  327791 equal" | grep -wE '^ *[0-9]+ equal' | sed 's/^ *//g' | cut -d ' ' -f1
echo "12327791 equal" | grep -wE '^ *[0-9]+ equal' | sed 's/^ *//g' | cut -d ' ' -f1
++ echo '  327791 equal'
++ grep -w '^ \{1,\}[0-9]* equal'
++ sed 's/^ *//g'
++ cut -d ' ' -f1
327791

++ echo '12327791 equal'
++ grep -w '^ \{1,\}[0-9]* equal'
++ sed 's/^ *//g'
++ cut -d ' ' -f1

++ echo '  327791 equal'
++ grep -wE '^ *[0-9]+ equal'
++ sed 's/^ *//g'
++ cut -d ' ' -f1
327791

++ echo '12327791 equal'
++ grep -wE '^ *[0-9]+ equal'
++ sed 's/^ *//g'
++ cut -d ' ' -f1
12327791

As you can see in the second execution, the old regular expression is unable to parse the large difference count number. The large difference count number is no longer padded with spaces, but the old regular expression demands at least one space. In the fourth execution we can see the new regular expression (not demanding a space), parsing the large number with success.

auanasgheps commented 2 months ago

Thank you! I've got a couple of PRs to merge before this one.

Have you had chance to verify it works well, right? It's not that easy to replicate such big changes!

desertwitch commented 2 months ago

Yes, I can confirm this works well both for equal/difference numbers below and above 8 digits. With the current (old) regular expression, the script fails with numbers above 8 digits.

auanasgheps commented 2 months ago

We need to re-base these changes, because as I anticipated I had a previous PR to merge. This PR added a feature to ignore files to decrease counts so is impacting this PR. I should be able to do this.

desertwitch commented 2 months ago

I adapted the PR around the new code, the conflict should be resolved now.

auanasgheps commented 2 months ago

Thank you, super fast!