filelock error

[captainstarlifter]

Yo Dr Earle!

Using fio 3.1 with your newest ezfio (but with longtime and shorttime set to 5 seconds (just to validate that everything is working. The test is also run with -u 1. This is just a sea trial!

It also failed using a ezfio version from last fall.

Is there a preferred version of fio?

Advice and counsel?

Thanks Earle!

---Random Preconditioning--- Random Preconditioning DONE DONE DONE Random Preconditioning DONE DONE DONE

---Sustained 4KB Random Read Tests by Number of Threads--- Sustained 4KB Random Read Tests by Number of Threads, Threads=1 43.53 11,143 88.6 Sustained 4KB Random Read Tests by Number of Threads, Threads=2 87.76 22,467 88.0 Sustained 4KB Random Read Tests by Number of Threads, Threads=4 177.59 45,462 87.3 Sustained 4KB Random Read Tests by Number of Threads, Threads=8 356.54 91,274 87.0 Sustained 4KB Random Read Tests by Number of Threads, Threads=16 709.67 181,676 87.5 Sustained 4KB Random Read Tests by Number of Threads, Threads=32 1,347.17 344,877 92.2 Sustained 4KB Random Read Tests by Number of Threads, Threads=64 2,442.68 625,325 101.6 Sustained 4KB Random Read Tests by Number of Threads, Threads=128 3,054.59 781,974 162.6 Sustained 4KB Random Read Tests by Number of Threads, Threads=256 3,053.83 781,782 325.3

---Sustained 4KB Random mixed 30% Write Tests by Threads--- Sustained 4KB Random mixed 30% Write Tests by Threads, Threads=1 54.96 14,071 88.8 Sustained 4KB Random mixed 30% Write Tests by Threads, Threads=2 113.50 29,056 87.0 Sustained 4KB Random mixed 30% Write Tests by Threads, Threads=4 225.70 57,779 87.9 Sustained 4KB Random mixed 30% Write Tests by Threads, Threads=8 438.29 112,203 90.5 Sustained 4KB Random mixed 30% Write Tests by Threads, Threads=16 818.62 209,568 97.1 Sustained 4KB Random mixed 30% Write Tests by Threads, Threads=32 1,488.00 380,927 107.1 Sustained 4KB Random mixed 30% Write Tests by Threads, Threads=64 2,474.59 633,495 129.0 Sustained 4KB Random mixed 30% Write Tests by Threads, Threads=128 3,546.50 907,904 178.1 Sustained 4KB Random mixed 30% Write Tests by Threads, Threads=256 3,926.65 1,005,223 319.8

---Sustained Perf Stability Test - 4KB Random 30% Write--- Sustained Perf Stability Test - 4KB Random 30% Write Runtime 00:00:17 ... FIO Error! fio /tmp/tmpRtscjI --output-format=json+ STDOUT:

STDERR: fio: filelock.c:182: fio_lock_file: Assertion `!trylock' failed. fio: filelock.c:182: fio_lock_file: Assertion !trylock' failed. fio: filelock.c:182: __fio_lock_file: Assertion!trylock' failed. fio: filelock.c:182: fio_lock_file: Assertion `!trylock' failed. fio: filelock.c:182: fio_lock_file: Assertion !trylock' failed. fio: filelock.c:182: __fio_lock_file: Assertion!trylock' failed. fio: filelock.c:182: fio_lock_file: Assertion `!trylock' failed. fio: filelock.c:182: fio_lock_file: Assertion !trylock' failed. fio: filelock.c:182: __fio_lock_file: Assertion!trylock' failed. fio: filelock.c:182: fio_lock_file: Assertion `!trylock' failed. fio: filelock.c:182: fio_lock_file: Assertion !trylock' failed. fio: filelock.c:182: __fio_lock_file: Assertion!trylock' failed. fio: filelock.c:182: fio_lock_file: Assertion `!trylock' failed. fio: filelock.c:182: fio_lock_file: Assertion !trylock' failed. fio: filelock.c:182: __fio_lock_file: Assertion!trylock' failed. fio: filelock.c:182: fio_lock_file: Assertion `!trylock' failed. fio: filelock.c:182: fio_lock_file: Assertion !trylock' failed. fio: filelock.c:182: __fio_lock_file: Assertion!trylock' failed. fio: filelock.c:182: fio_lock_file: Assertion `!trylock' failed. fio: filelock.c:182: fio_lock_file: Assertion !trylock' failed. fio: filelock.c:182: __fio_lock_file: Assertion!trylock' failed. fio: filelock.c:182: fio_lock_file: Assertion `!trylock' failed. fio: filelock.c:182: fio_lock_file: Assertion !trylock' failed. fio: filelock.c:182: __fio_lock_file: Assertion!trylock' failed. fio: filelock.c:182: fio_lock_file: Assertion `!trylock' failed. fio: filelock.c:182: fio_lock_file: Assertion !trylock' failed. fio: filelock.c:182: __fio_lock_file: Assertion!trylock' failed.

Exception in thread Thread-50: Traceback (most recent call last): File "/usr/lib64/python2.7/threading.py", line 812, in __bootstrap_inner self.run() File "/usr/lib64/python2.7/threading.py", line 765, in run self.target(*self.args, **self.__kwargs) File "./fast5.ezfio.py", line 1085, in JobWrapper val = o'cmdline' File "./fast5.ezfio.py", line 939, in o['runtime'])}) File "./fast5.ezfio.py", line 832, in RunTest raise FIOError(" ".join(cmdline), code, err, out) FIOError

Sustained Perf Stability Test - 4KB Random 30% Write ERROR ERROR ERROR ERROR DETECTED, ABORTING TEST RUN. You have new mail in /var/spool/mail/root fstsmtn10rt-1 ezfio-master #

[earlephilhower]

Howdy,

Check the ulimit on file locks. ulimit -x. If it's too small, you might end up with locking failures. Try ulimit -x unlimited

-EFP3

[earlephilhower]

Turns out it was a limit inside FIO, not an OS setting.

An FIO change was just merged, so if you use GIT head you should be all set. See https://github.com/axboe/fio/pull/838 for more details.