veeam / veeamsnap

Veeam Agent for Linux kernel module
GNU General Public License v2.0
72 stars 42 forks source link

kernel 4.18.0-305.19.1.el8_4.x86_64 issue #26

Closed badfiles closed 2 years ago

badfiles commented 3 years ago

On CentOS 8.4 (veeamsnap built from master)

4.18.0-305.19.1.el8_4.x86_64 #1 SMP Wed Sep 15 15:39:39 UTC 2021

Failed to create volume snapshot Failed to take volume snapshot Failed to perform backup Invalid argument Failed to execute IOCTL_TRACKING_ADD

[75451.576541] veeamsnap:main ================================================================================ [75451.576777] veeamsnap:cbt_prst | WRN | Persistent CBT parameters are not set [75451.577125] veeamsnap:main Loading [75451.578677] veeamsnap:main Version: 5.0.0.4318 [75451.579229] veeamsnap:main Author: Veeam Software Group GmbH [75451.579760] veeamsnap:main licence: GPL [75451.580256] veeamsnap:main description: Veeam Snapshot Kernel Module [75451.580815] veeamsnap:main zerosnapdata: 0 [75451.581357] veeamsnap:main debuglogging: 0 [75451.581898] veeamsnap:main snapstore_block_size_pow: 14 [75451.582704] veeamsnap:main change_tracking_block_size_pow: 18 [75451.583374] veeamsnap:main logdir: (null) [75451.583940] veeamsnap:main logmaxsize: 15728640 [75451.584567] veeamsnap:main cbtdata: (null) [75451.585135] veeamsnap:main fixflags: 0x0 [75451.585706] veeamsnap:main Compile for distributive: CentOS 8 [75451.586273] veeamsnap:ctrl_pipe Ctrl pipes initialization [75451.586842] veeamsnap:main Registering reboot notification [75451.587443] veeamsnap:main Module major [239] [75451.587986] veeamsnap:snapimage Snapshot image block device major 252 was registered [75451.588598] veeamsnap:ctrl_sysfs Create 'major' sysfs attribute [75451.589158] veeamsnap:ctrl_sysfs Create 'blkdev_notify' sysfs attribute [75514.231209] veeamsnap:ctrl_fops Collecting tracking devices: [75514.232096] veeamsnap:tracking There are no devices under tracking [75514.232547] veeamsnap:tracking Adding device [8:17] under tracking [75514.232962] veeamsnap:tracker Create tracker for device [8:17]. Start 0x800 sector, capacity 0x2744b17df sectors [75514.233384] veeamsnap:cbt_map CBT map create [75514.233796] veeamsnap:cbt_map Allocate CBT map of 20587915 [75514.244558] veeamsnap:cbt_map New CBT generation ID: [7456c87b-ad8f-4269-9802-dcee4169472d] [75514.326096] veeamsnap:tracker | ERR | Cannot make hook. make_request_fn is NULL [75514.326118] veeamsnap:blk Device [8:17] was frozen [75514.327497] veeamsnap:tracker | ERR | Failed to create tracker for device [8:17] [75514.327501] veeamsnap:blk Device [8:17] was unfrozen [75514.328903] veeamsnap:tracking | ERR | Failed to create tracker. errno=-22 [75514.328997] veeamsnap:blk Device [8:17] was frozen [75514.330360] veeamsnap:blk Device [8:17] was unfrozen

badfiles commented 3 years ago

however module built from https://github.com/SergeiShtepa/veeamsnap/tree/experimental works

Why the 8.4 patches are not in master yet?

HNGamingUK commented 3 years ago

however module built from https://github.com/SergeiShtepa/veeamsnap/tree/experimental works

Why the 8.4 patches are not in master yet?

On the readme the master states This module supports Linux kernels from 2.6.32 to 5.13.

Where the experimental is 5.14+

So although there has been a veeamsnap update recently (5.0.1) it currently only supports up to 5.13 kernel.

Update: I have noticed that your post states it's actually kernal 4.18... It should deffinately be supported by now considering the readme info.

badfiles commented 3 years ago

after updating veeam from repo i get broken server after starting the job

[  173.279282] veeamsnap: loading out-of-tree module taints kernel.
[  173.279442] veeamsnap: module verification failed: signature and/or required key missing - tainting kernel
[  173.282456] WARN veeamsnap:logging    Logging to file will be not performed
[  173.282492] veeamsnap:main       ================================================================================
[  173.282533] veeamsnap:main       Loading
[  173.282547] veeamsnap:main       Version: 5.0.1.4493
[  173.282563] veeamsnap:main       Author: Veeam Software Group GmbH
[  173.282581] veeamsnap:main       licence: GPL
[  173.282596] veeamsnap:main       description: Veeam Snapshot Kernel Module
[  173.282615] veeamsnap:main       zerosnapdata: 0
[  173.282646] veeamsnap:main       debuglogging: 0
[  173.282686] veeamsnap:main       snapstore_block_size_pow: 14
[  173.282705] veeamsnap:main       change_tracking_block_size_pow: 18
[  173.282733] veeamsnap:main       logdir: (null)
[  173.282755] veeamsnap:main       logmaxsize: 15728640
[  173.282778] veeamsnap:main       cbtdata: (null)
[  173.282794] veeamsnap:main       fixflags: 0x0
[  173.282809] veeamsnap:main       Compile for distributive: RHEL 8.0
[  173.282828] veeamsnap:ctrl_pipe  Ctrl pipes initialization
[  173.282845] veeamsnap:main       Registering reboot notification
[  173.282863] veeamsnap:main       Module major [239]
[  173.283329] veeamsnap:cbt_prst   | WRN | Persistent CBT parameters are not set
[  173.283334] veeamsnap:snapimage  Snapshot image block device major 252 was registered
[  173.283388] veeamsnap:ctrl_sysfs Create 'major' sysfs attribute
[  173.283408] veeamsnap:ctrl_sysfs Create 'blkdev_notify' sysfs attribute
filename:       /lib/modules/4.18.0-305.19.1.el8_4.x86_64/weak-updates/veeamsnap.ko
supported:      external
version:        5.0.1.4493
description:    Veeam Snapshot Kernel Module
author:         Veeam Software Group GmbH
license:        GPL
rhelversion:    8.3
srcversion:     B5BC38001779727F4234C77
depends:        
name:           veeamsnap
vermagic:       4.18.0-240.el8.x86_64 SMP mod_unload modversions 
sig_id:         PKCS#7
signer:         DigiCert EV Code Signing CA (SHA2)
sig_key:        0F:6F:90:B4:1A:F2:33:C6:4E:0A:F6:F8:B3:E5:6D:26
sig_hashalgo:   sha256

job hangs on step 'creating snapshot' aftet that FS in unusable, have to reset the server

[root@nova5 updates]# veeamconfig ui
[root@nova5 updates]# ls
bash: /bin/ls: Input/output error
ptide commented 3 years ago

Hi @badfiles ,

Support for RHEL 8.4 is implemented in the experimental branch and has been added to v5.0.1 release. Therefore, VAL 5.0.1.4493 should work with CentOS 8.4 just fine.

I suppose that the problem appears because you load 'veeamsnap' module using 'modprobe'.

Unfotnunately, it does not work for 'veeamsnap' on RH7 and RH8 because weak-updates script mistakenly thinks that the module from the previous kernel is compatible with newer kernels. Despite the earlier modules pass the whitelist checks, they still cause the kernel to crash. Apparently you've encountered exactly that issue:

filename: /lib/modules/4.18.0-305.19.1.el8_4.x86_64/weak-updates/veeamsnap.ko
...
rhelversion: 8.3
...
vermagic: 4.18.0-240.el8.x86_64 SMP mod_unload modversions

That is, the module that is suitable for 8.3 attempted to start on 8.4

It was RHEL 7.4 release when we saw such behaviour for the first time. Since then 'veeamsnap-kmod' module includes a special 'veeamsnap-loader' script that picks the right module version and loads it via the usage of 'insmod'. You can take a look at this tiny python script. 'veeamservice' service for RHEL 7 and RHEL 8 uses the script to load 'veeamsnap' module.

badfiles commented 3 years ago

@ptide thank you for the feedback, loading module with veeamsnap-loader solved the problem