gluster / glusterd2

[DEPRECATED] Glusterd2 is the distributed management framework to be used for GlusterFS.
GNU General Public License v2.0
167 stars 82 forks source link

cleanup resources if tests in e2e fails #1235

Open Madhu-1 opened 6 years ago

Madhu-1 commented 6 years ago

Observed behavior

if test cases in e2e fail, we are not cleaning up the resource, which leaves the mounted volume as it is

Expected/desired behavior

even if the test cases fails we need to clean up the used resources

Logs

=== RUN   TestSmartVolume/Smartvol_Without_Name
--- FAIL: TestSmartVolume (50.11s)
    smartvol_ops_test.go:263: Using temp dir: /tmp/gd2_func_test/TestSmartVolume022562123
    --- FAIL: TestSmartVolume/Smartvol_Distributed_Volume (3.13s)
        require.go:765: 
                Error Trace:    smartvol_ops_test.go:63
                Error:          Expected nil, but got: &errors.errorString{s:"Transaction step vol-delete.CleanBricks failed on peer 083e19d9-ea0c-4c9e-b339-bb57e76d9839 with error: exit status 5;   Failed to enumerate udev device list.\n  WARNING: Device for PV mV2M0a-iqMo-6THZ-ahXh-O3E0-Ul8H-uHb3af not found or rejected by a filter.\n  Aborting vg_write: No metadata areas to write to!\n\n"}
                Test:           TestSmartVolume/Smartvol_Distributed_Volume
    --- FAIL: TestSmartVolume/Smartvol_Replicate_2_Volume (2.28s)
        require.go:157: 
                Error Trace:    smartvol_ops_test.go:36
                                            smartvol_ops_test.go:87
                Error:          Not equal: 
                                expected: 0
                                actual  : 1
                Test:           TestSmartVolume/Smartvol_Replicate_2_Volume
    --- FAIL: TestSmartVolume/Smartvol_Replicate_3_Volume (2.76s)
        require.go:157: 
                Error Trace:    smartvol_ops_test.go:36
                                            smartvol_ops_test.go:111
                Error:          Not equal: 
                                expected: 0
                                actual  : 1
                Test:           TestSmartVolume/Smartvol_Replicate_3_Volume
    --- FAIL: TestSmartVolume/Smartvol_Arbiter_Volume (2.66s)
        require.go:157: 
                Error Trace:    smartvol_ops_test.go:36
                                            smartvol_ops_test.go:140
                Error:          Not equal: 
                                expected: 0
                                actual  : 1
                Test:           TestSmartVolume/Smartvol_Arbiter_Volume
    --- FAIL: TestSmartVolume/Smartvol_Disperse_Volume (5.44s)
        require.go:157: 
                Error Trace:    smartvol_ops_test.go:36
                                            smartvol_ops_test.go:166
                Error:          Not equal: 
                                expected: 0
                                actual  : 1
                Test:           TestSmartVolume/Smartvol_Disperse_Volume
    --- FAIL: TestSmartVolume/Smartvol_Distributed-Replicate_Volume (5.74s)
        require.go:157: 
                Error Trace:    smartvol_ops_test.go:36
                                            smartvol_ops_test.go:198
                Error:          Not equal: 
                                expected: 0
                                actual  : 1
                Test:           TestSmartVolume/Smartvol_Distributed-Replicate_Volume
    --- FAIL: TestSmartVolume/Smartvol_Distributed-Disperse_Volume (5.70s)
        require.go:157: 
                Error Trace:    smartvol_ops_test.go:36
                                            smartvol_ops_test.go:230
                Error:          Not equal: 
                                expected: 0
                                actual  : 1
                Test:           TestSmartVolume/Smartvol_Distributed-Disperse_Volume
    --- FAIL: TestSmartVolume/Smartvol_Without_Name (1.46s)
        require.go:157: 
                Error Trace:    smartvol_ops_test.go:36
                                            smartvol_ops_test.go:243
                Error:          Not equal: 
                                expected: 0
                                actual  : 1
                Test:           TestSmartVolume/Smartvol_Without_Name
=== RUN   TestSnapshot
--- FAIL: TestSnapshot (20.70s)
    require.go:765: 
            Error Trace:    snapshot_ops_test.go:49
            Error:          Expected nil, but got: &exec.ExitError{ProcessState:(*os.ProcessState)(0xc42097b8a0), Stderr:[]uint8{0x6c, 0x6f, 0x73, 0x65, 0x74, 0x75, 0x70, 0x3a, 0x20, 0x2f, 0x74, 0x6d, 0x70, 0x2f, 0x67, 0x64, 0x32, 0x5f, 0x66, 0x75, 0x6e, 0x63, 0x5f, 0x74, 0x65, 0x73, 0x74, 0x2f, 0x54, 0x65, 0x73, 0x74, 0x53, 0x6e, 0x61, 0x70, 0x73, 0x68, 0x6f, 0x74, 0x2f, 0x62, 0x72, 0x69, 0x63, 0x6b, 0x73, 0x2f, 0x31, 0x2f, 0x70, 0x61, 0x74, 0x63, 0x68, 0x79, 0x5f, 0x73, 0x6e, 0x61, 0x70, 0x5f, 0x6c, 0x6f, 0x6f, 0x70, 0x3a, 0x20, 0x66, 0x61, 0x69, 0x6c, 0x65, 0x64, 0x20, 0x74, 0x6f, 0x20, 0x73, 0x65, 0x74, 0x20, 0x75, 0x70, 0x20, 0x6c, 0x6f, 0x6f, 0x70, 0x20, 0x64, 0x65, 0x76, 0x69, 0x63, 0x65, 0x3a, 0x20, 0x50, 0x65, 0x72, 0x6d, 0x69, 0x73, 0x73, 0x69, 0x6f, 0x6e, 0x20, 0x64, 0x65, 0x6e, 0x69, 0x65, 0x64, 0xa}}
            Test:           TestSnapshot
=== RUN   TestVolume
=== RUN   TestVolume/CreateWithoutName
=== RUN   TestVolume/Create
=== RUN   TestVolume/Expand
=== RUN   TestVolume/Start
=== RUN   TestVolume/Mount
=== RUN   TestVolume/Status
=== RUN   TestVolume/Statedump
=== RUN   TestVolume/Stop
=== RUN   TestVolume/List
=== RUN   TestVolume/Info
=== RUN   TestVolume/Edit
=== RUN   TestVolume/VolumeFlags
=== RUN   TestVolume/Delete
=== RUN   TestVolume/Disperse
=== RUN   TestVolume/DisperseMount
=== RUN   TestVolume/DisperseDelete
=== RUN   TestVolume/testShdOnVolumeStartAndStop
=== RUN   TestVolume/SelfHeal
=== RUN   TestVolume/GranularEntryHeal
--- FAIL: TestVolume (25.53s)
    --- PASS: TestVolume/CreateWithoutName (0.04s)
    --- PASS: TestVolume/Create (0.04s)
    --- PASS: TestVolume/Expand (0.05s)
    --- PASS: TestVolume/Start (9.33s)
    --- FAIL: TestVolume/Mount (0.10s)
        require.go:765: 
                Error Trace:    volume_ops_test.go:455
                                            volume_ops_test.go:434
                                            utils_test.go:33
                Error:          Expected nil, but got: 0x10
                Test:           TestVolume/Mount
                Messages:       unmount failed: device or resource busy
    --- PASS: TestVolume/Status (0.06s)
mounted volume
127.0.0.1:testvol on /tmp/gd2_func_test/w1/run/gluster/gd2mount041209877 type fuse.glusterfs (ro,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)
prashanthpai commented 6 years ago

Related issue: https://github.com/gluster/glusterd2/issues/1223

atinmu commented 6 years ago

@Madhu-1 can this be closed?

Madhu-1 commented 6 years ago

Fixed in #1243

Madhu-1 commented 6 years ago

reopening as am issue some more issue in snapshot

=== RUN   TestSnapshot
=== RUN   TestSnapshot/Create
=== RUN   TestSnapshot/Activate
=== RUN   TestSnapshot/List
=== RUN   TestSnapshot/Mount
SIGQUIT: quit
PC=0x45f5a1 m=0 sigcode=0

goroutine 0 [idle]:
runtime.futex(0x15a0a60, 0x80, 0x0, 0x0, 0x0, 0xc000000000, 0x0, 0x0, 0x7fff9a3cd710, 0x40c2b2, ...)
    /usr/local/go/src/runtime/sys_linux_amd64.s:531 +0x21
runtime.futexsleep(0x15a0a60, 0x100000000, 0xffffffffffffffff)
    /usr/local/go/src/runtime/os_linux.go:46 +0x4b
runtime.notesleep(0x15a0a60)
    /usr/local/go/src/runtime/lock_futex.go:151 +0xa2
runtime.stopm()
    /usr/local/go/src/runtime/proc.go:2016 +0xe3
runtime.findrunnable(0xc00004aa00, 0x0)
    /usr/local/go/src/runtime/proc.go:2487 +0x4dc
runtime.schedule()
    /usr/local/go/src/runtime/proc.go:2613 +0x13a
runtime.park_m(0xc00024a300)
    /usr/local/go/src/runtime/proc.go:2676 +0xae
runtime.mcall(0x0)
    /usr/local/go/src/runtime/asm_amd64.s:299 +0x5b

goroutine 1 [chan receive, 7 minutes]:
testing.(*T).Run(0xc00030e400, 0xe1c9db, 0xc, 0xe50b90, 0x48ef01)
    /usr/local/go/src/testing/testing.go:879 +0x37a
testing.runTests.func1(0xc00011e000)
    /usr/local/go/src/testing/testing.go:1119 +0x78
testing.tRunner(0xc00011e000, 0xc000591da0)
    /usr/local/go/src/testing/testing.go:827 +0xbf
testing.runTests(0xc00000c040, 0x15912e0, 0xa, 0xa, 0xe20c5d)
    /usr/local/go/src/testing/testing.go:1117 +0x2aa
testing.(*M).Run(0xc00030c400, 0x0)
    /usr/local/go/src/testing/testing.go:1034 +0x165
github.com/gluster/glusterd2/e2e.TestMain(0xc00030c400)
    /home/mrajanna/workspace/heketi_pr/src/github.com/gluster/glusterd2/e2e/main_test.go:44 +0x20e
main.main()
    _testmain.go:58 +0x13d

goroutine 58 [chan receive]:
github.com/gluster/glusterd2/vendor/github.com/coreos/etcd/pkg/logutil.(*MergeLogger).outputLoop(0xc000262140)
    /home/mrajanna/workspace/heketi_pr/src/github.com/gluster/glusterd2/vendor/github.com/coreos/etcd/pkg/logutil/merge_logger.go:174 +0x39e
created by github.com/gluster/glusterd2/vendor/github.com/coreos/etcd/pkg/logutil.NewMergeLogger
    /home/mrajanna/workspace/heketi_pr/src/github.com/gluster/glusterd2/vendor/github.com/coreos/etcd/pkg/logutil/merge_logger.go:92 +0x80

goroutine 89 [chan receive]:
github.com/gluster/glusterd2/vendor/github.com/coreos/etcd/pkg/logutil.(*MergeLogger).outputLoop(0xc00026e2a0)
    /home/mrajanna/workspace/heketi_pr/src/github.com/gluster/glusterd2/vendor/github.com/coreos/etcd/pkg/logutil/merge_logger.go:174 +0x39e
created by github.com/gluster/glusterd2/vendor/github.com/coreos/etcd/pkg/logutil.NewMergeLogger
    /home/mrajanna/workspace/heketi_pr/src/github.com/gluster/glusterd2/vendor/github.com/coreos/etcd/pkg/logutil/merge_logger.go:92 +0x80

goroutine 90 [chan receive]:
github.com/gluster/glusterd2/vendor/github.com/coreos/etcd/pkg/logutil.(*MergeLogger).outputLoop(0xc00026e300)
    /home/mrajanna/workspace/heketi_pr/src/github.com/gluster/glusterd2/vendor/github.com/coreos/etcd/pkg/logutil/merge_logger.go:174 +0x39e
created by github.com/gluster/glusterd2/vendor/github.com/coreos/etcd/pkg/logutil.NewMergeLogger
    /home/mrajanna/workspace/heketi_pr/src/github.com/gluster/glusterd2/vendor/github.com/coreos/etcd/pkg/logutil/merge_logger.go:92 +0x80

goroutine 9 [chan receive, 9 minutes]:
github.com/gluster/glusterd2/glusterd2/daemon.init.0.func1()
    /home/mrajanna/workspace/heketi_pr/src/github.com/gluster/glusterd2/glusterd2/daemon/connection.go:72 +0xc6
created by github.com/gluster/glusterd2/glusterd2/daemon.init.0
    /home/mrajanna/workspace/heketi_pr/src/github.com/gluster/glusterd2/glusterd2/daemon/connection.go:71 +0x35

goroutine 1087 [syscall, 7 minutes]:

even if the tests fail we should umount the mounted volumes

/dev/mapper/patchy_snap_vg_1-brick_lvm on /abc/gd2_func_test/TestSnapshot/bricks9773367171/patchy_snap_mnt type xfs (rw,relatime,seclabel,nouuid,attr2,inode64,sunit=128,swidth=128,noquota)
/dev/mapper/patchy_snap_vg_2-brick_lvm on /abc/gd2_func_test/TestSnapshot/bricks9773367172/patchy_snap_mnt type xfs (rw,relatime,seclabel,nouuid,attr2,inode64,sunit=128,swidth=128,noquota)
/dev/mapper/patchy_snap_vg_3-brick_lvm on /abc/gd2_func_test/TestSnapshot/bricks9773367173/patchy_snap_mnt type xfs (rw,relatime,seclabel,nouuid,attr2,inode64,sunit=128,swidth=128,noquota)
/dev/mapper/patchy_snap_vg_4-brick_lvm on /abc/gd2_func_test/TestSnapshot/bricks9773367174/patchy_snap_mnt type xfs (rw,relatime,seclabel,nouuid,attr2,inode64,sunit=128,swidth=128,noquota)
127.0.0.1:/snaps/snaptest on /abc/gd2_func_test/TestSnapshot/Mount/mnt945030024 type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)