Client reading wrong data with parallel reads from multiple files

[uristdwarf]

When reading multiple files, when CacheExpirationTime is not 0, reading from a mount point can sometimes return wrong data.

Reproduction

1. Create a list of files from a SaunaFS mount point to read from
2. Using the script below, generate md5sum of the files in the SaunaFS mountpoint as background processes.

echo "Dropping caches"
echo 3 > /proc/sys/vm/drop_caches
echo "Calculating checkusms":
cat /replace/me/with/path/to/list/of/saunafs/files /
| while read -e line ; 
  do 
     md5sum "${line}" > $(mktemp /tmp/tmp.XXXXXXXXX.md5sum) &
  done;

while pgrep md5sum > /dev/null; do
  echo -n "$(pgrep md5sum | wc -l)..."
  sleep 1
done

sync

# Check how many threads created output
ls /tmp/tmp.*.md5sum | wc -l 

# Generate one large md5sum file
cat  /tmp/tmp.*.md5sum > /tmp/parallel.md5sum.result.v2

# Clean up individual md5sum
rm /tmp/tmp.*.md5sum

echo "Dropping caches"
echo 3 > /proc/sys/vm/drop_caches

echo "Sleeping 10 seconds"
sleep 10

echo "Checking md5sums file"
md5sum --check /tmp/parallel.md5sum.result.v2 | grep FAIL

Workaround:

Run echo "CacheExpirationTime=0" > /saunafs/mountpoint on existing mount points, and use -o cacheExpirationTime=0 on when mounting

[uristdwarf]

Fixed by 52bb86a1029a87d32bdf52e7508f0dbf456c98bb