tools/biolatency: Add support for blktrace events

[Caturra000]

On WSL2 with the kernel 6.4.8, biolatency.py and biolatency.py -Q did not work.

In this case, the kernel kprobe functions were inlined and block:block_io_{start|done} tracepoints were missing, thus no condition was matched.

This fix adds support for block:block_bio_queue and block:block_rq_complete tracepoints, corresponding to the blktrace Q event (Queue) and C event (Complete) respectively. Now biolatency.py and the -Q option are compatible with the kernel.

References: events and tracepoints.

[yonghong-song]

I am not sure how block:block_bio_queue tracepoint works here. In latest bpf-next, I see submit_bio_noacct_nocheck trace_block_bio_queue

submit_bio_noacct_nocheck is called in a few other places but looks like it is for some special occasions like blk_zone_wplug_bio_work(), blk_throtl_dispatch_work_fn(), submit_bio_noacct().

Maybe I missed something here?

[Caturra000]

I think submit_bio_noacct() -> submit_bio_noacct_nocheck() is a common IO routine, not for special occasions only.

For example, we can trace the dd if=/dev/sda of=/dev/null bs=4k command using bpftrace -e 'tracepoint:block:block_bio_queue { print(kstack); }' and ... block:block_io_start ...:

// block_bio_queue
        submit_bio_noacct_nocheck+396
        submit_bio_noacct+354
        submit_bio+178
        mpage_readahead+241
        blkdev_readahead+21
        read_pages+149
        page_cache_ra_unbounded+359
        page_cache_ra_order+719
        ondemand_readahead+540
        page_cache_async_ra+88
        filemap_get_pages+776
        filemap_read+247
        blkdev_read_iter+109
        vfs_read+600
        ksys_read+115
        __x64_sys_read+25
        x64_sys_call+6874
        do_syscall_64+127

// block_io_start
        blk_mq_submit_bio+1338
        __submit_bio+179
        submit_bio_noacct_nocheck+316
        submit_bio_noacct+354
        submit_bio+178
        mpage_readahead+241
        blkdev_readahead+21
        read_pages+149
        page_cache_ra_unbounded+359
        page_cache_ra_order+719
        ondemand_readahead+540
        page_cache_async_ra+88
        filemap_get_pages+776
        filemap_read+247
        blkdev_read_iter+109
        vfs_read+600
        ksys_read+115
        __x64_sys_read+25
        x64_sys_call+6874
        do_syscall_64+127

(Results from my real PC running kernel 6.8, not WSL 6.4.8.)

We can see submit_bio_noacct_nocheck() exists in both paths.

[yonghong-song]

Ok, I checked the call stack you described in the above and compared them to the 6.9 kernel source. Looks like indeed the stack trace you mentioned are possible.