search

TritonDataCenter / smartos-live

For more information, please see http://smartos.org/ For any questions that aren't answered there, please join the SmartOS discussion list: https://smartos.topicbox.com/groups/smartos-discuss
1.57k stars 246 forks source link

ZFS data prefetch broken #764

Closed Evgueni-Gavrilov closed 6 years ago

Evgueni-Gavrilov commented 6 years ago

[root@qct1 ~]# kstat -m zfs | grep -i prefetch demand_hit_predictive_prefetch 0 prefetch_data_hits 0 prefetch_data_misses 0 prefetch_metadata_hits 47 prefetch_metadata_misses 201

[root@qct1 ~]# echo "::zfs_params" | mdb -k | grep -i prefetch zfs_prefetch_disable = 0x0 zfs_no_scrub_prefetch = 0x1

[root@qct1 ~]# w 07:51:16 up 1 day(s), 1:04, 1 user, load average: 0.26, 0.32, 0.35

[root@qct1 ~]# mdb -kwe zfs_prefetch_disable/W0t0 zfs_prefetch_disable: 0 = 0x0

[root@qct1 ~]# arcstat -f pread,ph% 1 10 pread ph% 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0

[root@qct1 ~]# kstat -m zfs | grep -i prefetch demand_hit_predictive_prefetch 0 prefetch_data_hits 0 prefetch_data_misses 0 prefetch_metadata_hits 47 prefetch_metadata_misses 207

[root@qct1 ~]# head -1 /etc/release SmartOS 20180203T031130Z x86_64

plitc commented 6 years ago 
[root@assg15-labor ~]# kstat -m zfs | grep -i prefetch
        demand_hit_predictive_prefetch  0
        prefetch_data_hits              0
        prefetch_data_misses            0
        prefetch_metadata_hits          311
        prefetch_metadata_misses        192
[root@assg15-labor ~]#
[root@assg15-labor ~]# head -1 /etc/release
                      SmartOS 20180314T152546Z x86_64
[root@assg15-labor ~]#

https://github.com/ass-a2s/smartos-live

based on joyent commits up to 13.03.2018

jclulow commented 6 years ago

I'm not sure if this is specific to SmartOS, per se. Looking at the same statistics on an OI hipster VM running current illumos-gate master without modifications:

jclulow@oi0 ~ manta:jpc 
 $ head -1 /etc/release 
             OpenIndiana Hipster 2017.10 (powered by illumos)
jclulow@oi0 ~ manta:jpc 
 $ uname -v
rti-master-0-gb13f152e9f
jclulow@oi0 ~ manta:jpc 
 $ kstat -p zfs:::*prefetch*
zfs:0:arcstats:demand_hit_predictive_prefetch   0
zfs:0:arcstats:prefetch_data_hits       0
zfs:0:arcstats:prefetch_data_misses     0
zfs:0:arcstats:prefetch_metadata_hits   344163
zfs:0:arcstats:prefetch_metadata_misses 17332
Evgueni-Gavrilov commented 6 years ago

the same issue (no data prefetch; metadata only) with the latest release:

[root@qct0 ~]# kstat -p zfs:::prefetch zfs:0:arcstats:demand_hit_predictive_prefetch 0 zfs:0:arcstats:prefetch_data_hits 0 zfs:0:arcstats:prefetch_data_misses 0 zfs:0:arcstats:prefetch_metadata_hits 170 zfs:0:arcstats:prefetch_metadata_misses 151

[root@qct0 ~]# head -1 /etc/release SmartOS 20180315T080815Z x86_64

plitc commented 6 years ago

no issues on 20180118T013028Z

Evgueni-Gavrilov commented 6 years ago

kstat -p zfs:::prefetch

zfs:0:arcstats:demand_hit_predictive_prefetch 0 zfs:0:arcstats:prefetch_data_hits 0 zfs:0:arcstats:prefetch_data_misses 0 zfs:0:arcstats:prefetch_metadata_hits 220 zfs:0:arcstats:prefetch_metadata_misses 417

head -1 /etc/release

                  SmartOS 20180412T025853Z x86_64
Evgueni-Gavrilov commented 6 years ago

fixed @47b8d4b8:

kstat -p zfs:::prefetch

zfs:0:arcstats:demand_hit_predictive_prefetch 7053 zfs:0:arcstats:prefetch_data_hits 8330 zfs:0:arcstats:prefetch_data_misses 23689 zfs:0:arcstats:prefetch_metadata_hits 466 zfs:0:arcstats:prefetch_metadata_misses 716