search

autotest / tp-libvirt

Test Provider for Libvirt and related virtualization backends
Other
33 stars 169 forks source link

passthrough: add robustness test suite #5853

Closed smitterl closed 3 weeks ago

smitterl commented 2 months ago

Add test suite that will execute configured actions on passthrough devices and VMs.

Scenarios (edge coverage)
VM
all should end with "check_present" if applicable
edge coverage: 1
RUNNING -> attach -> pause resume 1 dev all dev
RUNNING -> attach -> reboot 1 dev all dev
RUNNING -> attach -> detach attach 1 dev all dev
RUNNING -> attach -> detach attach -> reboot 1 dev all dev
RUNNING -> attach -> shutdown start -> attach 1 dev all dev
STOPPED -> attach -> start -> RUNNING -> detach attach 1 dev all dev
STOPPED -> attach -> start -> RUNNING -> reboot 1 dev all dev
STOPPED -> attach -> start -> RUNNING -> shutdown start 1 dev all dev
STOPPED -> attach -> start -> RUNNING -> destroy start 1 dev all dev
STOPPED -> attach -> start -> RUNNING -> pause resume 1 dev all dev
RUNNING -> attach -> 10x detach attach 1 dev all dev
STOPPED -> attach -> start -> 10x detach attach reboot 1 dev all dev
RUNNING -> attach -> attach 1 dev type all supported dev types
RUNNING -> attach -> attach -> reboot 1 dev type all supported dev types
RUNNING -> attach -> attach -> pause resume 1 dev type all supported dev types
RUNNING -> attach -> attach -> detach attach -> detach attach 1 dev type all supported dev types
RUNNING -> attach -> attach -> reboot -> detach attach -> detach attach 1 dev type all supported dev types
Device
Mediated devices need to be restarted, only possible when not used in VM
RUNNING -> attach -> detach -> restart dev -> attach 1 dev all dev
Migrate device between VMs
(RUNNING0 -> attach detach) -> (RUNNING1 attach) 1 dev all dev
(RUNNING0 -> attach detach) -> (RUNNING1 attach detach) -> (RUNNING0 attach) 1 dev all dev

The MdevHandler classes had to be updated to retain state, so that reusing them would create the same conditions as using the first time both for device creation (ids and driver overrides) and checks.

Apply black, inspekt and isort tools for code quality.

Depends on https://github.com/avocado-framework/avocado-vt/pull/3975

smitterl commented 2 months ago

@chunfuwen @legoater Please help review.

smitterl commented 2 months ago

Convert to draft as tests and regression coverage still running.

smitterl commented 2 months ago

Test runs

JOB ID     : 81e7c7444e843215916c8582c9a839e0a4b1754a                                                                                                                                                                                                                                                                                                                                                                                                                                                                       
JOB LOG    : /var/log/avocado/job-results/job-2024-08-21T12.38-81e7c74/job.log                                                                                                                                                                                                                                                                                                                                                                                                                                              
 (001/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.start_stop_ap: ERROR: Command '/bin/virsh nodedev-create /tmp/xml_utils_temp_4b7cj3lr.xml' failed.\nstdout: b'\n'\nstderr: b"error: Failed to open file '/tmp/xml_utils_temp_4b7cj3lr.xml': No such file or directory\n"\nadditional_info: None (6.53 s)                                                                                                                                                                                         
 (002/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.start_stop_ism: PASS (6.32 s)                                                                                                                                                                                                                                                                                                                                                                                                                    
 (003/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.start_stop_roce: PASS (6.21 s)                                                                                                                                                                                                                                                                                                                                                                                                                   
 (004/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.start_stop_ccw: PASS (7.50 s)                                                                                                                                                                                                                                                                                                                                                                                                                    
 (005/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.start_destroy_vm: PASS (19.60 s)                                                                                                                                                   
 (006/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.start_stop_all: PASS (8.24 s)                                                                                                                                                      
 (007/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.hotplug_hotunplug_all: PASS (50.22 s)                                                                                                                                              
 (008/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.hotplug_check_all: PASS (24.99 s)                                                                                                                                                  
 (009/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.hotplug_all_distribute_over_two: PASS (25.54 s)                                                                                                                                    
 (010/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_pause_resume.ap: PASS (15.79 s)                                                                                                                                  
 (011/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_pause_resume.ccw: PASS (16.21 s)                                                                                                                                 
 (012/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_pause_resume.ism: PASS (16.55 s)                                                                                                                                 
 (013/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_pause_resume.roce: PASS (19.25 s)                                                                                                                                
 (014/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_pause_resume.all: PASS (25.79 s)                                                                                                                                 
 (015/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_reboot.ap: PASS (50.01 s)                                                                                                                                        
 (016/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_reboot.ccw: PASS (50.62 s)                                                                                                                                       
 (017/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_reboot.ism: PASS (48.84 s)                                                                                                                                       
 (018/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_reboot.roce: PASS (52.88 s)                                                                                                                                      
 (019/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_reboot.all: PASS (58.56 s)                                                                                                                                       
 (020/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_attach.ap: PASS (14.26 s)                                                                                                                                 
 (021/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_attach.ccw: PASS (15.93 s)                                                                                                                                
 (022/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_attach.ism: PASS (15.27 s)                                                                                                                                
 (023/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_attach.roce: PASS (25.18 s)                                                                                                                               
 (024/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_attach.all: PASS (33.51 s)                                                                                                                                
 (025/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_attach_reboot.ap: PASS (48.42 s)                                                                                                                          
 (026/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_attach_reboot.ccw: PASS (50.37 s)                                                                                                                         
 (027/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_attach_reboot.ism: PASS (49.34 s)                                                                                                                         
 (028/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_attach_reboot.roce: PASS (59.22 s)                                                                                                                        
 (029/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_attach_reboot.all: PASS (67.73 s)                                                                                                                         
 (030/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_shutdown_start_attach.ap: PASS (20.69 s)                                                                                                                         
 (031/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_shutdown_start_attach.ccw: PASS (21.33 s)                                                                                                                        
 (032/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_shutdown_start_attach.ism: PASS (20.13 s)                                                                                                                        
 (033/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_shutdown_start_attach.roce: PASS (30.02 s)                                                                                                                       
 (034/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_shutdown_start_attach.all: PASS (36.53 s)                                                                                                                        
 (035/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_detach_attach.ap: PASS (15.57 s)                                                                                                                           
 (036/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_detach_attach.ccw: PASS (16.15 s)                                                                                                                          
 (037/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_detach_attach.ism: PASS (14.99 s)                                                                                                                          
 (038/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_detach_attach.roce: PASS (26.60 s)                                                                                                                         
 (039/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_detach_attach.all: PASS (63.15 s)                                                                                                                          
 (040/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_reboot.ap: PASS (50.08 s)                                                                                                                                  
 (041/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_reboot.ccw: PASS (50.61 s)                                                                                                                                 
 (042/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_reboot.ism: PASS (49.26 s)                                                                                                                                 
 (043/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_reboot.roce: PASS (57.11 s)                                                                                                                                
 (044/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_reboot.all: PASS (63.70 s)                                                                                                                                 
 (045/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_shutdown_start.ap: PASS (22.05 s)                                                                                                                          
 (046/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_shutdown_start.ccw: PASS (21.64 s)                                                                                                                         
 (047/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_shutdown_start.ism: PASS (20.57 s)                                                                                                                         
 (048/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_shutdown_start.roce: PASS (35.43 s)                                                                                                                        
 (049/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_shutdown_start.all: PASS (41.39 s)                                                                                                                         
 (050/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_destroy_start.ap: PASS (24.68 s)                                                                                                                           
 (051/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_destroy_start.ccw: PASS (25.83 s)                                                                                                                          
 (052/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_destroy_start.ism: PASS (25.00 s)                                                                                                                          
 (053/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_destroy_start.roce: PASS (40.37 s)                                                                                                                         
 (054/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_destroy_start.all: PASS (47.18 s)                                                                                                                          
 (055/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_pause_resume.ap: PASS (15.69 s)                                                                                                                            
 (056/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_pause_resume.ccw: PASS (16.42 s)                                                                                                                           
 (057/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_pause_resume.ism: PASS (15.59 s)                                                                                                                           
 (058/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_pause_resume.roce: PASS (23.07 s)                                                                                                                          
 (059/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_pause_resume.all: PASS (29.49 s)                                                                                                                           
 (060/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_10_times_detach_attach.ap: PASS (15.24 s)                                                                                                                        
 (061/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_10_times_detach_attach.ccw: PASS (16.73 s)                                                                                                                       
 (062/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_10_times_detach_attach.ism: PASS (15.17 s)                                                                                                                       
 (063/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_10_times_detach_attach.roce: PASS (115.27 s)                                                                                                                     
 (064/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_10_times_detach_attach.all: PASS (130.49 s)                                                                                                                      
 (065/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_10_times_detach_attach_reboot.ap: PASS (362.27 s)                                                                                                          
 (066/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_10_times_detach_attach_reboot.ccw: PASS (361.62 s)                                                                                                         
 (067/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_10_times_detach_attach_reboot.ism: PASS (358.98 s)                                                                                                         
 (068/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_10_times_detach_attach_reboot.roce: PASS (459.01 s)                                                                                                        
 (069/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_stopped_attach_start_10_times_detach_attach_reboot.all: PASS (470.16 s)                                                                                                         
 (070/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_attach.ccw: PASS (19.08 s)                                                                                                                                       
 (071/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_attach.ism: PASS (15.45 s)                                                                                                                                       
 (072/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_attach.roce: PASS (26.47 s)                                                                                                                                      
 (073/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_attach.all: PASS (36.61 s)                                                                                                                                       
 (074/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_attach_reboot.ccw: PASS (52.70 s)                                                                                                                                
 (075/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_attach_reboot.ism: PASS (50.37 s)                             
 (076/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_attach_reboot.roce: PASS (60.74 s)                           
 (077/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_attach_reboot.all: PASS (71.31 s)           
 (078/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_attach_pause_resume.ccw: PASS (19.46 s)      
 (079/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_attach_pause_resume.ism: PASS (16.83 s)      
 (080/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_attach_pause_resume.roce: PASS (26.21 s)
 (081/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_attach_pause_resume.all: PASS (36.91 s)
 (082/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_attach_detach_attach_detach_attach.ccw: PASS (17.64 s)
 (083/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_attach_detach_attach_detach_attach.ism: PASS (16.38 s)
 (084/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_attach_detach_attach_detach_attach.roce: PASS (46.30 s)
 (085/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_attach_detach_attach_detach_attach.all: PASS (57.22 s)
 (086/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_attach_reboot_detach_attach_detach_attach.ccw: PASS (52.76 s)
 (087/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_attach_reboot_detach_attach_detach_attach.ism: PASS (50.73 s)
 (088/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_attach_reboot_detach_attach_detach_attach.roce: PASS (81.52 s)
 (089/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_attach_reboot_detach_attach_detach_attach.all: PASS (92.40 s)
 (090/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_mdevrestart_attach.ap: PASS (15.62 s)
 (091/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_mdevrestart_attach.ccw: PASS (16.78 s)
 (092/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_mdevrestart_attach.all: PASS (34.38 s)                     
 (093/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_vm2_attach.ap: PASS (17.37 s)                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                           
 (094/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_vm2_attach.ccw: PASS (17.39 s)                             
 (095/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_vm2_attach.ism: PASS (16.89 s)                                                                                  
 (096/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_vm2_attach.roce: PASS (26.96 s)                                                                                 
 (097/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_vm2_attach.all: PASS (35.78 s)                                                                                  
 (098/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_vm2_attach_detach_vm_attach.ap: PASS (16.47 s)                                                                  
 (099/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_vm2_attach_detach_vm_attach.ccw: PASS (16.85 s)                                                                 
 (100/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_vm2_attach_detach_vm_attach.ism: PASS (15.56 s)                                                                 
 (101/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_vm2_attach_detach_vm_attach.roce: PASS (34.25 s)                                                                
 (102/102) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_detach_vm2_attach_detach_vm_attach.all: PASS (46.31 s)                                                                 
RESULTS    : PASS 101 | ERROR 1 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0                                                                                                                                 
JOB HTML   : /var/log/avocado/job-results/job-2024-08-21T12.38-81e7c74/results.html                                                                                                                                 
JOB TIME   : 5294.58 s

The first test failure happened because __init__.py/get_nodedev_xml initially only returned the file path, that apparently lead to the tmp file to be deleted early in some occasions. Now return the full object seems to fix this issue (I assume because the tmp-cleanup kicked when an object wasn't referenced anymore):

 (1/1) type_specific.io-github-autotest-libvirt.passthrough.robustness.start_stop_ap: PASS (5.98 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML   : /var/log/avocado/job-results/job-2024-08-22T04.19-025f0e9/results.html
JOB TIME   : 6.50 s
No python imaging library installed. Screendump and Windows guest BSOD detection are disabled. In order to enable it, please install python-imaging or the equivalent for your distro.
No python imaging library installed. PPM image conversion to JPEG disabled. In order to enable it, please install python-imaging or the equivalent for your distro.
JOB ID     : 8f610bd0d429b294241459b49b808af68eb59aa1
JOB LOG    : /var/log/avocado/job-results/job-2024-08-22T04.19-8f610bd/job.log
 (1/1) type_specific.io-github-autotest-libvirt.passthrough.robustness.start_stop_ap: PASS (6.23 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML   : /var/log/avocado/job-results/job-2024-08-22T04.19-8f610bd/results.html
JOB TIME   : 6.76 s
No python imaging library installed. Screendump and Windows guest BSOD detection are disabled. In order to enable it, please install python-imaging or the equivalent for your distro.
No python imaging library installed. PPM image conversion to JPEG disabled. In order to enable it, please install python-imaging or the equivalent for your distro.
JOB ID     : 650d45d61ac6f9e336ce16452fba5fddd926ec22
JOB LOG    : /var/log/avocado/job-results/job-2024-08-22T04.19-650d45d/job.log
 (1/1) type_specific.io-github-autotest-libvirt.passthrough.robustness.start_stop_ap: PASS (6.45 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML   : /var/log/avocado/job-results/job-2024-08-22T04.19-650d45d/results.html
JOB TIME   : 6.99 s
No python imaging library installed. Screendump and Windows guest BSOD detection are disabled. In order to enable it, please install python-imaging or the equivalent for your distro.
No python imaging library installed. PPM image conversion to JPEG disabled. In order to enable it, please install python-imaging or the equivalent for your distro.
JOB ID     : 55344fb8277443d758b82076c4cfbb939405c807
JOB LOG    : /var/log/avocado/job-results/job-2024-08-22T04.19-55344fb/job.log
 (1/1) type_specific.io-github-autotest-libvirt.passthrough.robustness.start_stop_ap: PASS (6.19 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML   : /var/log/avocado/job-results/job-2024-08-22T04.19-55344fb/results.html
JOB TIME   : 6.71 s
No python imaging library installed. Screendump and Windows guest BSOD detection are disabled. In order to enable it, please install python-imaging or the equivalent for your distro.
No python imaging library installed. PPM image conversion to JPEG disabled. In order to enable it, please install python-imaging or the equivalent for your distro.
JOB ID     : a93e242a4d55039dff359654f09ad8e43595d738
JOB LOG    : /var/log/avocado/job-results/job-2024-08-22T04.19-a93e242/job.log
 (1/1) type_specific.io-github-autotest-libvirt.passthrough.robustness.start_stop_ap: PASS (6.32 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML   : /var/log/avocado/job-results/job-2024-08-22T04.19-a93e242/results.html
JOB TIME   : 6.83 s
No python imaging library installed. Screendump and Windows guest BSOD detection are disabled. In order to enable it, please install python-imaging or the equivalent for your distro.
No python imaging library installed. PPM image conversion to JPEG disabled. In order to enable it, please install python-imaging or the equivalent for your distro.
JOB ID     : 09d607f4981663066d70b99465e454de76cd3d24
JOB LOG    : /var/log/avocado/job-results/job-2024-08-22T04.19-09d607f/job.log
 (1/1) type_specific.io-github-autotest-libvirt.passthrough.robustness.start_stop_ap: PASS (6.42 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML   : /var/log/avocado/job-results/job-2024-08-22T04.19-09d607f/results.html
JOB TIME   : 6.96 s
No python imaging library installed. Screendump and Windows guest BSOD detection are disabled. In order to enable it, please install python-imaging or the equivalent for your distro.
No python imaging library installed. PPM image conversion to JPEG disabled. In order to enable it, please install python-imaging or the equivalent for your distro.
JOB ID     : af1a93bcd78e5a9cc01f06a503dfa1a115985788
JOB LOG    : /var/log/avocado/job-results/job-2024-08-22T04.20-af1a93b/job.log
 (1/1) type_specific.io-github-autotest-libvirt.passthrough.robustness.start_stop_ap: PASS (6.59 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML   : /var/log/avocado/job-results/job-2024-08-22T04.20-af1a93b/results.html
JOB TIME   : 7.11 s
...

I'm now also running existing tests that use the provider/vfio code to confirm no regression was introduced.

smitterl commented 2 months ago 
JOB ID     : 138c3f577d1beef5d6caa5c24df00e9e4a1a7332
JOB LOG    : /var/log/avocado/job-results/job-2024-08-22T08.12-138c3f5/job.log
 (01/14) type_specific.io-github-autotest-libvirt.virtual_disks.dasd.read_native_partition_table: PASS (20.80 s)
 (02/14) type_specific.io-github-autotest-libvirt.virsh.nodedev_persistence.mdev.ccw: PASS (4.81 s)
 (03/14) type_specific.io-github-autotest-libvirt.libvirt_ap_passthrough.autostart: PASS (53.42 s)
 (04/14) type_specific.io-github-autotest-libvirt.virsh.nodedev_dumpxml.chain.device_type_dasd: PASS (3.96 s)
 (05/14) type_specific.io-github-autotest-libvirt.virsh.nodedev_dumpxml.chain.device_type_css: PASS (4.27 s)
 (06/14) type_specific.io-github-autotest-libvirt.libvirt_ap_passthrough.hw_hotplug: PASS (27.96 s)
 (07/14) type_specific.io-github-autotest-libvirt.libvirt_ccw_passthrough.device_removal: ERROR: VM 'avocado-vt-vm1' failed to start: error: Failed to start domain 'avocado-vt-vm1'\nerror: device not found: mediated device '312b30cd-9fd8-418f-970f-2f9d7760b544' not found(exit status: 1) (26.37 s)
 (08/14) type_specific.io-github-autotest-libvirt.virsh.crypto_nodedev_create_destroy.positive: PASS (35.10 s)
 (09/14) type_specific.io-github-autotest-libvirt.libvirt_ap_passthrough.hostdev_hotplug: PASS (12.37 s)
 (10/14) type_specific.io-github-autotest-libvirt.libvirt_ap_passthrough.hostdev_coldplug: PASS (23.83 s)
 (11/14) type_specific.io-github-autotest-libvirt.libvirt_pci_passthrough.ism.happy_path: PASS (17.74 s)
 (12/14) type_specific.io-github-autotest-libvirt.libvirt_pci_passthrough.ism.reboot: PASS (51.47 s)
 (13/14) type_specific.io-github-autotest-libvirt.libvirt_pci_passthrough.ism.reboot_start: PASS (61.55 s)
 (14/14) type_specific.io-github-autotest-libvirt.libvirt_pci_passthrough.ism.connect: PASS (27.59 s)
RESULTS    : PASS 13 | ERROR 1 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML   : /var/log/avocado/job-results/job-2024-08-22T08.12-138c3f5/results.html
JOB TIME   : 372.51 s

The failing device_removal is a know to be brittle test which I can't reproduce and not related to this change (it doesn't actually use the mdev_handlers) , executed several times:

JOB ID     : 0901cfdba5d3152d6f25f80578f5ea74501c05f6
JOB LOG    : /var/log/avocado/job-results/job-2024-08-22T08.20-0901cfd/job.log
 (1/1) type_specific.io-github-autotest-libvirt.libvirt_ccw_passthrough.device_removal: PASS (41.36 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML   : /var/log/avocado/job-results/job-2024-08-22T08.20-0901cfd/results.html
JOB TIME   : 41.73 s
smitterl commented 2 months ago

additional single test run to cover the latest changes based on review

JOB ID     : a6d60d575928b8e2d982b2b800741b5cb26da071
JOB LOG    : /var/log/avocado/job-results/job-2024-09-04T04.44-a6d60d5/job.log
 (1/1) type_specific.io-github-autotest-libvirt.passthrough.robustness.vm_running_attach_pause_resume.all: PASS (25.54 s)
RESULTS    : PASS 1 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB HTML   : /var/log/avocado/job-results/job-2024-09-04T04.44-a6d60d5/results.html
JOB TIME   : 26.03 s