search

axboe / fio

Flexible I/O Tester
GNU General Public License v2.0
5.21k stars 1.26k forks source link

userspace_reap only works temporarily #704

Closed KipIngram closed 5 years ago

KipIngram commented 5 years ago

Hi. I've been trying to optimize small block latency on an IBM FlashSystem storage unit. I find that if I engage userspace_reap=1, along with the other parameter that requires, I see about a 1-2 microsecond improvement in single transaction latency. But this improvement only persists for about 20-30 seconds. Then I see a very brief (order of 1 second) period of poor performance, followed by a "recovery" back to "almost but not quite" the initial good performance.

I'm looking for ideas as to what could be happening there at 20-30 seconds; I'd like to make the userspace_reap benefits persist throughout my whole test without issue.

Is there some "resource" that userspace_reap "consumes," and once it's gone things have to operate differently? I played with various options for memory allocation and so on, but didn't identify anything that relieved this issue.

Thanks!

sitsofe commented 5 years ago

@KipIngram as this isn't so much an issue in fio and more of a "How do I/Why is?" question it is better aimed at the fio mailing list (note that the list only accepts plain text emails). Could you try re-asking your question there?

axboe commented 5 years ago

I'll try and take a look, but the aio user ring is basically useless, so I won't be surprised if there's an issue with it.

Please share what your job file and storage looks like, any good report should come with a way to reproduce the issue.

sitsofe commented 5 years ago

@KipIngram are you able to provide the informations Jens was asking for?

axboe commented 5 years ago

I'd be more inclined to just kill it, the current aio ring user interface is the worst and hopefully we'll have something better soon. Closing this report, as we're more than 2 months in and no reply from the reporter.