search

olopez32 / ganeti

Automatically exported from code.google.com/p/ganeti
0 stars 0 forks source link

DRBD cannot grow disk due to syncing #765

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
QA fails with this error. Either this is a bug, or we should add some delay in 
the QA calls to make sure the syncing is done before we attempt to resize the 
disks.

(Not sure if this only appears in 2.11, if necessary target it for an earlier 
version.)

Please use labels and text to provide additional information.

<<<< 2014-03-19 21:45:05.403805 start [TestInstanceGrowDisk] gnt-instance 
grow-disk
Checking if instance 'INST2' is not running
172.25.78.68 is unreachable
Command: ssh NODE1 'gnt-instance grow-disk INST2 0 512M'
Wed Mar 19 21:45:09 2014 Growing disk 0 of instance 'INST2' by 512M to 2.5G
Wed Mar 19 21:45:12 2014  - INFO: Waiting for instance INST2 to sync disks
Wed Mar 19 21:45:12 2014  - INFO: - device disk/0: 10.90% done, 9s remaining 
(estimated)
Wed Mar 19 21:45:22 2014  - INFO: Instance INST2's disks are in sync
Command: ssh NODE1 'gnt-instance grow-disk --absolute INST2 0 2G'
Failure: prerequisites not met for this operation:
error type: wrong_state, error details:
Requested size (2.0G) is smaller than current disk size (2.5G)
Command: ssh NODE1 'gnt-instance grow-disk --absolute INST2 0 3072'
Wed Mar 19 21:45:28 2014 Growing disk 0 of instance 'INST2' by 512M to 3.0G
Failure: command execution error:
Grow request failed to node NODE1: Failed to grow block device: drbd0: resize 
failed: 0: Failure: (130) Resize not allowed during resync.
---- FAILED [TestInstanceGrowDisk] gnt-instance grow-disk: Command 
'gnt-instance grow-disk --absolute INST2 0 3072' on node NODE2 failed, exit 
code 1
>>>> 2014-03-19 21:45:31.497388 time=0:00:26.093583 [TestInstanceGrowDisk] 
gnt-instance grow-disk
---- BLOCK FAILED [RunInstanceTests] Create and exercise instances: Command 
'gnt-instance grow-disk --absolute INST2 0 3072' on node NODE1 failed, exit 
code 1
^^^^ BLOCK 2014-03-19 21:45:31.497577 time=0:35:21.948791 [RunInstanceTests] 
Create and exercise instances
Traceback (most recent call last):
  File "./ganeti-qa.py", line 1011, in <module>
    main()
  File "/buildbot/qa-drbd84/qa-drbd84-full/build/ganeti/rapi/client.py", line 245, in wrapper
    return fn(*args, **kwargs)
  File "./ganeti-qa.py", line 1006, in main
    RunQa()
  File "./ganeti-qa.py", line 908, in RunQa
    RunTestBlock(RunInstanceTests)
  File "./ganeti-qa.py", line 145, in RunTestBlock
    return fn(*args, **kwargs)
  File "./ganeti-qa.py", line 805, in RunInstanceTests
    RunCommonInstanceTests(instance, inodes)
  File "./ganeti-qa.py", line 357, in RunCommonInstanceTests
    RunTestIf(["instance-grow-disk"], qa_instance.TestInstanceGrowDisk, instance)
  File "./ganeti-qa.py", line 122, in RunTestIf
    RunTest(fn, *args, **kwargs)
  File "./ganeti-qa.py", line 101, in RunTest
    retval = fn(*args, **kwargs)
  File "/buildbot/qa-drbd84/qa-drbd84-full/build/qa/qa_utils.py", line 717, in wrapper
    result = fn(*args, **kwargs)
  File "/buildbot/qa-drbd84/qa-drbd84-full/build/qa/qa_instance.py", line 697, in TestInstanceGrowDisk
    str(int_size + 2 * int_grow)])
  File "/buildbot/qa-drbd84/qa-drbd84-full/build/qa/qa_utils.py", line 199, in AssertCommand
    _AssertRetCode(rcode, fail, cmdstr, nodename)
  File "/buildbot/qa-drbd84/qa-drbd84-full/build/qa/qa_utils.py", line 169, in _AssertRetCode
    (cmdstr, nodename, rcode))
qa_error.Error: Command 'gnt-instance grow-disk --absolute INST2 0 3072' on 
node NODE1 failed, exit code 1
program finished with exit code 1
elapsedTime=9233.208098

Original issue reported on code.google.com by hel...@google.com on 20 Mar 2014 at 8:19

GoogleCodeExporter commented 9 years ago 
[deleted comment]
GoogleCodeExporter commented 9 years ago 
Just a status update: I am trying to reproduce this issue with a full QA run on 
that very same cluster, but the first one did not reveal anything. Next update 
in 4-5 hours...

Original comment by r...@google.com on 20 Mar 2014 at 5:49

GoogleCodeExporter commented 9 years ago

Original comment by hel...@google.com on 16 Apr 2014 at 10:53

GoogleCodeExporter commented 9 years ago 
As this hasn't appeared since, it looks like a DRBD quirk, therefore we're 
postponing the issue.

Original comment by pud...@google.com on 24 Jun 2014 at 12:41