Write throughput degradation on second file created after a format, if in subdirectory

[liem-to]

Using fio to benchmark exfat performance for my video record application. After performing a format, the second file created has around 20% slower write throughput than the first one, if both file are in different subdirectories of my drive.

Attaching my fio script below. Steps to reproduce:

1. Format drive: mkexfatfs -s 2048 -n cfast /dev/sdb1

2. Set the root directory for fio script create subdirectory and run first test. export MEDIA_MOUNT=/media/liem/cfast mkdir -p ${MEDIA_MOUNT}/dir1 fio --section=subdir_1 exfat.job Results:

   WRITE: bw=2032MiB/s (2130MB/s)

3. Create second subdirectory and run second test. mkdir -p ${MEDIA_MOUNT}/dir2 fio --section=subdir_2 exfat.job Results:

   WRITE: bw=1611MiB/s (1690MB/s)

4. Writing to second file in second subdirectory is much slower 1690MB/s vs 2130MB/s.

5. Now reformat and run test again but keep files in same directory. mkexfatfs -s 2048 -n cfast /dev/sdb1

6. Write test to first file fio --section=no_subdir_1 exfat.job Results:

   WRITE: bw=2036MiB/s (2135MB/s)

7. Write test to second file fio --section=no_subdir_2 exfat.job Results:

   WRITE: bw=2002MiB/s (2099MB/s)

8. Pretty much the same throughput in this test 2099 MB/s vs 2135 MB/s.

9. My test environment: Ubuntu 18.04.1 LTS on HP Z230 desktop Intel i7. kernel: 4.15.0-70-generic Sony cFAST card connected with Xcellon cFAST card reader formatted with exFAT. Using tag 2.2.0-3arter97 of exfat-linux.

10. My fio test script exfat.job:

; fio test sequential write jobs ; MEDIA_MOUNT is the root directory of the mounted card, e.g. /media/liem/cfast

[global] rw=write ioengine=sync numjobs=1 fallocate=none blocksize=4MB filesize=2GB create_on_open=1

[subdir_1] filename=${MEDIA_MOUNT}/dir1/test1.bin

[subdir_2] filename=${MEDIA_MOUNT}/dir2/test2.bin

[no_subdir_1] filename=${MEDIA_MOUNT}/test1.bin

[no_subdir_2] filename=${MEDIA_MOUNT}/test2.bin

[arter97]

Your write speed indicates that the data is not being flushed to your cFAST card, which means it's being written to RAM.

I have a feeling that your RAM is not enough to store everything until the 2nd run, hence making it slower.

Before each runs, can you try sync; echo 3 > /proc/sys/vm/drop_caches and post the results again?

Thanks.

[liem-to]

Thanks for your reply.

I tried your suggestions and got the same results. Write throughput drop off on the second record if in subdirectories 2114 MB/s to 1686 MB/s, but no drop off if in the same root directory, 2101 MB/s and 2116 MB/s.

Here is what I did: Format my cFAST card.

mkdir -p $MEDIA_MOUNT/dir1
sync
echo 3 | sudo tee -a /proc/sys/vm/drop_caches 
fio --section=subdir_1 exfat.job

WRITE: bw=2016MiB/s (2114MB/s)

mkdir -p $MEDIA_MOUNT/dir2
sync
echo 3 | sudo tee -a /proc/sys/vm/drop_caches 
fio --section=subdir_2 exfat.job

WRITE: bw=1608MiB/s (1686MB/s)

In the no subdirectory case: Format cFAST card.

sync
echo 3 | sudo tee -a /proc/sys/vm/drop_caches 
fio --section=no_subdir_1 exfat.job

WRITE: bw=2004MiB/s (2101MB/s)

sync
echo 3 | sudo tee -a /proc/sys/vm/drop_caches 
fio --section=no_subdir_2 exfat.job

WRITE: bw=2018MiB/s (2116MB/s)

FYI, I have 32 GB of RAM on this PC. Here is the output of free after an fio run and after flushing the page cache.

free
              total        used        free      shared  buff/cache   available
Mem:       32687656     2608196    26875148      107096     3204312    29657064
Swap:       2097148           0     2097148

echo 3 | sudo tee -a /proc/sys/vm/drop_caches 
free
              total        used        free      shared  buff/cache   available
Mem:       32687656     2602328    29429332      107096      655996    29732704
Swap:       2097148           0     2097148

[arter97]

The master branch is now rebased to a different base, and all existing previous issues need to be re-evaluated.

https://github.com/arter97/exfat-linux/releases/tag/5.8-1arter97

Please re-open this if you believe the new base is still susceptible to this issue.