cifsd-team / ksmbd

ksmbd kernel server(SMB/CIFS server)
152 stars 23 forks source link

cifsd: null pointer on openwrt at ksmbd_vfs_kern_path causes kernel oops #425

Open Bernie opened 4 years ago

Bernie commented 4 years ago

This occurs on ath79 architecture with openwrt v19.07.3 with ksmbd-server version 3.2.1-1.

Stack trace follows:

Time: 1593919964.182145
Modules:        ksmbd@84180000+36409    ath9k@86ee0000+18ffb    ath9k_common@86f70000+2f6d      pppoe@86dfc000+23f0     ppp_async@86dfa000+1af0 ath9k_hw@86e80000+5556c ath10k_pci@86d60000+85cf   ath10k_core@86e00000+5de74      ath@86c68000+49f3       pppox@876d7000+56a      ppp_generic@86cb0000+58e2       nf_conntrack_ipv6@86c5c000+1200 mac80211@86d80000+71ba8    iptable_nat@86c86000+2f0        ipt_REJECT@86c54000+3f0 ipt_MASQUERADE@86c5f000+2d0     cfg80211@86cc0000+39134 xt_time@86c57000+710    xt_tcpudp@86c4b000+710  xt_tcpmss@86c43000+450     xt_statistic@86c32000+370       xt_state@86c15000+2f0   xt_nat@86c4d000+630     xt_multiport@86c45000+510       xt_mark@86c50000+2d0    xt_mac@86c37000+290     xt_limit@86c5a000+4d0      xt_length@86c52000+2f0  xt_hl@86c36000+350      xt_ecn@86c51000+570     xt_dscp@86c2e000+410    xt_conntrack@86c4f000+930       xt_comment@86c31000+210 xt_TCPMSS@8769c000+af0     xt_REDIRECT@86c27000+2f0        xt_LOG@86c2f000+330     xt_HL@876cd000+550      xt_FLOWOFFLOAD@86c33000+c00     xt_DSCP@86c30000+5f0    xt_CLASSIFY@877db000+270   ums_usbat@86c08000+2050 ums_sddr55@86c1e000+1750        ums_sddr09@86c10000+23d0        ums_karma@877d7000+6d0  ums_jumpshot@86c20000+f90       ums_isd200@86c18000+1700  ums_freecom@86c1b000+8d0 ums_datafab@87710000+12f0       ums_cypress@87669000+a50        ums_alauda@86c0c000+22b0        slhc@86c06000+145b      nf_reject_ipv4@877d6000+883     nf_nat_redirect@877d9000+5fb       nf_nat_masquerade_ipv4@877eb000+6fc     nf_conntrack_ipv4@877d0000+1110 nf_nat_ipv4@8779f000+d51        nf_nat@877e8000+24cb    nf_log_ipv4@8766d000+d30   nf_flow_table_hw@87719000+870   nf_flow_table@87714000+363f     nf_defrag_ipv6@8771c000+12be    nf_defrag_ipv4@8769d000+416     nf_conntrack_rtcache@877c9000+a70       nf_conntrack@877f0000+e1c5 iptable_mangle@877c8000+3d0     iptable_filter@877b6000+2b0     ipt_ECN@87761000+590    ip_tables@877c4000+276d crc_ccitt@87757000+40b  compat@877b0000+1977       ledtrig_usbport@8778f000+ae0    xt_set@877b2000+1b50    ip_set_list_set@8775c000+1c50   ip_set_hash_netportnet@877b8000+6c10    ip_set_hash_netport@877a8000+6170       ip_set_hash_netnet@87798000+6770   ip_set_hash_netiface@877a0000+6130      ip_set_hash_net@87790000+5b70   ip_set_hash_mac@87764000+28d0   ip_set_hash_ipportnet@87788000+66d0     ip_set_hash_ipportip@87780000+54b0 ip_set_hash_ipport@87778000+5090        ip_set_hash_ipmark@87770000+4dd0        ip_set_hash_ip@87768000+4cf0    ip_set_bitmap_port@87746000+17b0  ip_set_bitmap_ipmac@87758000+1b70        ip_set_bitmap_ip@8774e000+1ab0  ip_set@87750000+586a    nfnetlink@8773c000+1187 nf_log_ipv6@87742000+ff0        nf_log_common@8773f000+a4fip6table_mangle@87735000+4f0     ip6table_filter@8771e000+2b0    ip6_tables@87738000+26e1        ip6t_REJECT@87728000+430        x_tables@87730000+36af  nf_reject_ipv6@87725000+9a8ip6_gre@87708000+38b0   ip_gre@876d8000+3435    gre@875b8000+8d3        ip6_tunnel@876d0000+588f        tunnel6@876fe000+73e    ip_tunnel@876b8000+2e10 tun@876c0000+5891       multipath@876bc000+1370    raid456@876e0000+1cdce  libcrc32c@8767d000+297  async_raid6_recov@8767e000+421  async_pq@8764a000+706   async_xor@8765c000+449  xor@876b6000+189b       raid6_pq@876a0000+15430    async_memcpy@87656000+2ae       async_tx@875a4000+439   raid10@87670000+b130    raid1@87640000+7750     raid0@87648000+1ee0     md_mod@87680000+1b67d   nls_utf8@875ad000+370      sha512_generic@875b4000+24d9    sha256_generic@875b0000+28d9    seqiv@87599000+730      jitterentropy_rng@875a6000+1c78 drbg@875a8000+44b0      md5@8759a000+1290  md4@87597000+d10        hmac@87589000+9f0       ghash_generic@87586000+790      gf128mul@8758a000+17fe  gcm@8758c000+2720       ecb@87584000+590        des_generic@87590000+494b  ctr@8763e000+cb0        cmac@87622000+870       ccm@874ec000+1ff0       uas@87408000+2d80       usb_storage@87470000+a44b       ehci_platform@8741e000+13d0     sd_mod@87438000+7240       scsi_mod@87420000+16b12 ehci_hcd@87480000+8ad7  gpio_button_hotplug@87fd0000+1af0       ext4@87500000+5f595     mbcache@87fba000+c6e    jbd2@87fe0000+cb81      usbcore@87440000+20823     nls_base@87fd4000+1420  usb_common@87fb1000+9f7 crc16@87fc1000+407      aead@87fb2000+e61       crypto_null@87fb6000+c22        cryptomgr@87fc0000+799  crc32c_generic@87fb0000+590        crypto_hash@87fbc000+2752

<< unrelated logs excluded >>

<3>[  256.758812] ksmbd: kill_server_store:480: kill command received
<3>[  314.919136] ksmbd: __rpc_method:85: Unsupported RPC:
<3>[  314.924479] ksmbd: create_smb2_pipe:1762: Unable to open RPC pipe: -22
<1>[  314.959247] CPU 0 Unable to handle kernel paging request at virtual address 00000000, epc == 804104d0, ra == 8014fcb8
<4>[  314.970425] Oops[#1]:
<4>[  314.972780] CPU: 0 PID: 3207 Comm: kworker/0:4 Not tainted 4.14.180 #0
<4>[  314.979600] Workqueue: ksmbd-io ksmbd_ipc_init [ksmbd]
<4>[  314.984906] task: 854f4000 task.stack: 85516000
<4>[  314.989580] $ 0   : 00000000 80660000 00000000 00000003
<4>[  314.994983] $ 4   : 00000000 00000001 85517d24 00000000
<4>[  315.000386] $ 8   : 00000001 00000000 00001000 00000029
<4>[  315.005780] $12   : 0022652d 0dc52760 ffffffff 00000200
<4>[  315.011183] $16   : 00000001 85517d24 01400000 80510000
<4>[  315.016586] $20   : 00000000 84957b00 856bd000 841adb00
<4>[  315.021989] $24   : 80509040 1d0e2ce0
<4>[  315.027392] $28   : 85516000 85517c60 80510000 8014fcb8
<4>[  315.032796] Hi    : 0022652d
<4>[  315.035762] Lo    : 0dc52760
<4>[  315.038737] epc   : 804104d0 strlen+0xc/0x20
<4>[  315.043160] ra    : 8014fcb8 getname_kernel+0x2c/0xf4
<4>[  315.048368] Status: 1100dc03      KERNEL EXL IE
<4>[  315.052687] Cause : 00800008 (ExcCode 02)
<4>[  315.056826] BadVA : 00000000
<4>[  315.059793] PrId  : 00019750 (MIPS 74Kc)
<4>[  315.063841] Modules linked in: ksmbd ath9k ath9k_common pppoe ppp_async ath9k_hw ath10k_pci ath10k_core ath pppox ppp_generic nf_conntrack_ipv6 mac80211 iptable_nat ipt_REJECT ipt_MASQUERADE cfg80211 xt_time xt_tcpudp xt_tcpmss xt_statistic xt_state xt_nat xt_multiport xt_mark xt_mac xt_limit xt_length xt_hl xt_ecn xt_dscp xt_conntrack xt_comment xt_TCPMSS xt_REDIRECT xt_LOG xt_HL xt_FLOWOFFLOAD xt_DSCP xt_CLASSIFY ums_usbat ums_sddr55 ums_sddr09 ums_karma ums_jumpshot ums_isd200 ums_freecom ums_datafab ums_cypress ums_alauda slhc nf_reject_ipv4 nf_nat_redirect nf_nat_masquerade_ipv4 nf_conntrack_ipv4 nf_nat_ipv4 nf_nat nf_log_ipv4 nf_flow_table_hw nf_flow_table nf_defrag_ipv6 nf_defrag_ipv4 nf_conntrack_rtcache nf_conntrack iptable_mangle iptable_filter ipt_ECN ip_tables crc_ccitt compat ledtrig_usbport
<4>[  315.137502]  xt_set ip_set_list_set ip_set_hash_netportnet ip_set_hash_netport ip_set_hash_netnet ip_set_hash_netiface ip_set_hash_net ip_set_hash_mac ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_hash_ipport ip_set_hash_ipmark ip_set_hash_ip ip_set_bitmap_port ip_set_bitmap_ipmac ip_set_bitmap_ip ip_set nfnetlink nf_log_ipv6 nf_log_common ip6table_mangle ip6table_filter ip6_tables ip6t_REJECT x_tables nf_reject_ipv6 ip6_gre ip_gre gre ip6_tunnel tunnel6 ip_tunnel tun multipath raid456 libcrc32c async_raid6_recov async_pq async_xor xor raid6_pq async_memcpy async_tx raid10 raid1 raid0 md_mod nls_utf8 sha512_generic sha256_generic seqiv jitterentropy_rng drbg md5 md4 hmac ghash_generic gf128mul gcm ecb des_generic ctr cmac ccm uas usb_storage ehci_platform sd_mod scsi_mod ehci_hcd gpio_button_hotplug
<4>[  315.211072]  ext4 mbcache jbd2 usbcore nls_base usb_common crc16 aead crypto_null cryptomgr crc32c_generic crypto_hash
<4>[  315.222142] Process kworker/0:4 (pid: 3207, threadinfo=85516000, task=854f4000, tls=00000000)
<4>[  315.230936] Stack : 014000c0 87c6a000 8369ba80 802e8f5c 00000001 85517d24 00000000 00000001
<4>[  315.239573]         85517d24 80150210 85459a00 8369ba80 852dfda5 000005a8 85459ae4 85517db8
<4>[  315.248210]         00004000 86893c00 00000000 84184ad8 8369ba80 0000009c 000005a8 85459ae4
<4>[  315.256848]         01080020 00004000 85459a00 8034cfe8 841b05a8 85517d04 86893c00 800b4054
<4>[  315.265485]         000005a8 841ad880 86893c00 87722700 87722780 86893c00 856bf700 84957b00
<4>[  315.274113]         ...
<4>[  315.276642] Call Trace:
<4>[  315.279164] [<804104d0>] strlen+0xc/0x20
<4>[  315.283223] [<8014fcb8>] getname_kernel+0x2c/0xf4
<4>[  315.288081] [<80150210>] kern_path+0x1c/0x48
<4>[  315.292538] [<84184ad8>] ksmbd_vfs_kern_path+0x38/0x1cc [ksmbd]
<4>[  315.298680] [<8419265c>] ksmbd_queue_work+0x1c88/0x2590 [ksmbd]
<4>[  315.304803] Code: 10000002  00801025  24420001 <80430000> 1460fffd  00000000  03e00008  00441023  00852821
<4>[  315.314882]
<4>[  315.317031] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 1, skipped old beacon
<4>[  315.324635] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 2, skipped old beacon
<4>[  315.332290] ath10k_pci 0000:00:00.0: SWBA overrun on vdev 0, skipped old beacon
<4>[  315.340637] ---[ end trace 80c769b33d9e33bb ]---

Error occurs when a client makes an initial request on a share, before any authentication. On the client side, request times out, followed by disconnect from network as the router goes down and reboots.

namjaejeon commented 4 years ago

@Bernie Can you reproduce this issue with 100% frequency ? If yes, can you share tcpdump after reproduction ?

Bernie commented 4 years ago

Yes, 100%. The client is KDE Dolphin, version 17.12.3. Not only does it occur 100% of the time, but the client apparently will send the offending packet repeatedly while it is still open. Issue occurs when listing shares from the server. To reproduce, I open up the client, select "Samba Shares" and then type the server address in the top location field.

On the first attempt, it listed the available shares before crashing, on other attempts, it crashes before the list is generated.

Attached, please find the applicable pcap from tcpdump. I filtered on traffic from ports 128, 139, and 445. Please let me know if I missed anything in the capture.

Github won't let me attach the pcap directly, I had to gzip it. Please let me know if you have any issues with the attachment. cifsd.pcap.gz

namjaejeon commented 4 years ago

Cool, Thanks for your help. maybe, there is race condition issue between kill server and client connection. I wll take a look.

namjaejeon commented 4 years ago

One more request, can you share tcpdump when client connect with samba ? I think that this client doesn't send tree connect request about share.

sergey-senozhatsky commented 4 years ago

Hmm, something odd. I don't see empty rpc methods in the capture dump, yet here we go

<3>[  314.919136] ksmbd: __rpc_method:85: Unsupported RPC:
<3>[  314.924479] ksmbd: create_smb2_pipe:1762: Unable to open RPC pipe: -22

Update.

No, there are empty payloads. Packets ## 51, 52, 53, 54

Filename:
   Blob Length: 0

@namjaejeon I guess we should fail such requests (with zero filename blob lenght)

E.g.


diff --git a/fs/cifsd/unicode.c b/fs/cifsd/unicode.c
index 1dc7bd141794..526f741bec4d 100644
--- a/fs/cifsd/unicode.c
+++ b/fs/cifsd/unicode.c
@@ -279,6 +279,9 @@ smb_strndup_from_utf16(const char *src, const int maxlen,
        int len, ret;
        char *dst;

+       if (!maxlen)
+               return ERR_PTR(-EINVAL);
+
        if (is_unicode) {
                len = smb_utf16_bytes((__le16 *) src, maxlen, codepage);
                len += nls_nullsize(codepage);
namjaejeon commented 4 years ago

@sergey-senozhatsky Ah, The empty file name means root, i.e. share path. So it is no problem. The problem seems to be share->path is NULL in smb2_get_info_filesystem(). When analyzing packets, This client only sends tree_connect for IPC and does not send tree connect requests for share. And the tree id of the request for get info filesystem is IPC. I wonder if smb2_get_info_filesystem_pipe should be implemented like smb2_get_info_file_pipe(). And this request is FS_SIZE_INFORMATION, which mean we need to set filesystem statfs informations. but this is IPC share, not share in local filesystem. So I want to know how samba fill response about this request(smb2 get info filesystem - FS_SIZE_INFORMATION).

sergey-senozhatsky commented 4 years ago

At the same time we certainly don't expect empty rpc method

namjaejeon commented 4 years ago

@sergey-senozhatsky Let me check.

@Bernie Is it possible that you provide tcpdump with samba to me ?

Bernie commented 4 years ago

Unfortunately, I never had samba running on this router and was hoping to avoid doing so.

I'll see if I can get it running on a different machine if it'll help.

Andy2244 commented 4 years ago

Unfortunately, I never had samba running on this router and was hoping to avoid doing so.

If you have the space (~9 MB) all you need is install the samba4-server + luci package, the luci/UCI interface is nearly identically to ksmbd. Also both can be installed at the same time, just make sure only one is actually running.

namjaejeon commented 4 years ago

@Andy2244 Thanks for your info.

@Bernie I will try to install Dolphin file manager on my ubuntu PC. If I can not reproduce it, I will request it to you again.

namjaejeon commented 4 years ago

@Bernie I installed dolphin in my ubuntu, but It seems to not support SMB connection. I am grateful if you provide a tcpdump after setting up samba on your target as Andy guided.

namjaejeon commented 4 years ago

@Bernie What is your linux distribution installed dolphin by default ? CentOS ? or Mint ? maybe, I should install your environment to reproduce this issue.

Neustradamus commented 3 years ago

@Bernie: Any news? Have you looked all previous comments?