cholcombe973 / gluster-charm

Juju Gluster Charm
Other
8 stars 1 forks source link

"Skipping invalid block device" when using a scsi disk + juju storage with tagged maas disks #31

Open dshcherb opened 7 years ago

dshcherb commented 7 years ago

Tried to do a test on qemu virtual machines with virtio-scsi disks tagged in MAAS.

ubuntu-q87:/tmp$ juju --version
2.2-beta4-xenial-amd64

ubuntu@maas:~$ maas maas version read
Success.
Machine-readable output follows:
{"subversion": "bzr6054-0ubuntu1~16.04.1", "version": "2.2.0", "capabilities": ["networks-management", "static-ipaddresses", "ipv6-deployment-ubuntu", "devices-management", "storage-deployment-ubuntu", "network-deployment-ubuntu", "bridging-interface-ubuntu", "bridging-automatic-ubuntu", "authenticate-api"]}

ubuntu-q87:/tmp$ juju create-storage-pool glusterpool maas tags=jstorage

ubuntu-q87:/tmp$ juju storage-pools 
Name         Provider  Attrs
glusterpool  maas      tags=jstorage
loop         loop      
maas         maas      
rootfs       rootfs    
tmpfs        tmpfs     

ubuntu-q87:/tmp$ juju deploy ./gluster-charm -n 3 --storage brick='glusterpool'
Deploying charm "local:xenial/gluster-0".

# for better error reporting of mk2fs
ubuntu-q87:/tmp$ juju config gluster filesystem_type
ext4

ubuntu-q87:/tmp$ juju status
Model    Controller  Cloud/Region  Version    SLA
default  vmaas       vmaas         2.2-beta4  unsupported

App      Version  Status   Scale  Charm    Store  Rev  OS      Notes
gluster  3.10.2   blocked      3  gluster  local    0  ubuntu  

Unit        Workload  Agent  Machine  Public address  Ports  Message
gluster/0   blocked   idle   0        10.10.101.65           No bricks found
gluster/1*  blocked   idle   1        10.10.101.63           No bricks found
gluster/2   blocked   idle   2        10.10.101.64           No bricks found

Machine  State    DNS           Inst id  Series  AZ       Message
0        started  10.10.101.65  xcgtpc   xenial  default  Deployed
1        started  10.10.101.63  rynftc   xenial  default  Deployed
2        started  10.10.101.64  rw7fq3   xenial  default  Deployed

Relation  Provides  Consumes  Type

The block device seems to be skipped (see the log below) "Skipping invalid block device: "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1""

ubuntu@maas-xenial5:~$ readlink /dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1
../../sdb
ubuntu@maas-xenial5:~$ lsblk 
NAME   MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda      8:0    0  64G  0 disk 
└─sda1   8:1    0  64G  0 part /
sdb      8:16   0  64G  0 disk 

ubuntu@maas-xenial5:~$ lsscsi 
[2:0:0:0]    disk    QEMU     QEMU HARDDISK    2.5+  /dev/sda 
[2:0:0:1]    disk    QEMU     QEMU HARDDISK    2.5+  /dev/sdb 
unit-gluster-0: 19:31:15 INFO unit.gluster/0.juju-log Checking for new devices
unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log Checking for ephemeral unmount
unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log Gathering list of manually specified brick devices
unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log List of manual storage brick devices: []
unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log Gathering list of juju storage brick devices
unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log List of juju storage brick devices: ["/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1"]
unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log Checking if "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1" is a block device
unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log Skipping invalid block device: "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1"
unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log storage devices: []
unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log Usable brick paths: []
unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log .juju-persistent-config saved.  Wrote 445 bytes
unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log .juju-persistent-config saved.  Wrote 445 bytes
unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log .juju-persistent-config saved.  Wrote 445 bytes
unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log No upgrade requested
unit-gluster-0: 19:31:15 INFO unit.gluster/0.config-changed Volume info get command failed with error: Volume test does not exist
unit-gluster-0: 19:31:15 INFO unit.gluster/0.config-changed 
unit-gluster-0: 19:31:16 INFO unit.gluster/0.server-relation-changed Volume info get command failed with error: Volume test does not exist
unit-gluster-0: 19:31:16 INFO unit.gluster/0.server-relation-changed 
unit-gluster-1: 19:31:17 DEBUG unit.gluster/1.juju-log server:0: I am the leader: 0
unit-gluster-1: 19:31:18 INFO unit.gluster/1.juju-log server:0: Loading config
unit-gluster-1: 19:31:18 DEBUG unit.gluster/1.juju-log server:0: peer list: [UUID: bc6b036f-2bae-4f2a-8f61-dd04fe1446ae Hostname: 10.10.101.63 Status: Connected]
unit-gluster-2: 19:31:18 INFO unit.gluster/2.server-relation-changed Volume info get command failed with error: Volume test does not exist
unit-gluster-2: 19:31:18 INFO unit.gluster/2.server-relation-changed 
unit-gluster-0: 19:31:18 INFO unit.gluster/0.server-relation-changed Volume info get command failed with error: Volume test does not exist
unit-gluster-0: 19:31:18 INFO unit.gluster/0.server-relation-changed 
unit-gluster-1: 19:31:18 DEBUG unit.gluster/1.juju-log server:0: relation-list output: gluster/0

unit-gluster-1: 19:31:18 DEBUG unit.gluster/1.juju-log server:0: Adding in related_units: [Relation { name: "gluster", id: 0 }]
unit-gluster-1: 19:31:18 DEBUG unit.gluster/1.juju-log server:0: Adding 10.10.101.65 to cluster
unit-gluster-1: 19:31:18 DEBUG unit.gluster/1.juju-log server:0: Gluster peer probe was successful
unit-gluster-1: 19:31:18 INFO unit.gluster/1.server-relation-changed Volume info get command failed with error: Volume test does not exist
unit-gluster-1: 19:31:18 INFO unit.gluster/1.server-relation-changed 
unit-gluster-1: 19:31:18 INFO unit.gluster/1.juju-log server:0: Creating volume test
unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Waiting for all peers to enter the Peer in Cluster status
unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Got peer status: [UUID: c5642b67-77bf-4669-af24-30817c3ff55d Hostname: 10.10.101.65 Status: peer in cluster]
unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Checking for ephemeral unmount
unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Gathering list of manually specified brick devices
unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: List of manual storage brick devices: []
unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Gathering list of juju storage brick devices
unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: List of juju storage brick devices: ["/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1"]
unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Checking if "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1" is a block device
unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Skipping invalid block device: "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1"
unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: storage devices: []
unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Usable brick paths: []
unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Volume is none
unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Not enough peers to satisfy the replication level for the Gluster volume.  Waiting for more peers to join.
unit-gluster-1: 19:31:19 INFO unit.gluster/1.juju-log server:0: Waiting for more peers
unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Waiting for all peers to enter the Peer in Cluster status
unit-gluster-1: 19:31:19 ERROR unit.gluster/1.juju-log server:0: Hook failed with error: Some("Mount failed. Please check the log file for more details.\n")
unit-gluster-1: 19:31:19 INFO unit.gluster/1.server-relation-changed Volume info get command failed with error: Volume test does not exist

Adding this device to a list of brick devices without redeploying does not work:

ubuntu-q87:/tmp$ juju config gluster brick_devices=/dev/sdb

unit-gluster-1: 20:49:03 INFO unit.gluster/1.juju-log Checking for new devices
unit-gluster-0: 20:49:03 INFO unit.gluster/0.juju-log Checking for new devices
unit-gluster-2: 20:49:03 INFO unit.gluster/2.juju-log Checking for new devices
unit-gluster-1: 20:49:03 DEBUG unit.gluster/1.juju-log Checking for ephemeral unmount
unit-gluster-0: 20:49:03 DEBUG unit.gluster/0.juju-log Checking for ephemeral unmount
unit-gluster-2: 20:49:03 DEBUG unit.gluster/2.juju-log Checking for ephemeral unmount
unit-gluster-1: 20:49:03 DEBUG unit.gluster/1.juju-log Gathering list of manually specified brick devices
unit-gluster-2: 20:49:03 DEBUG unit.gluster/2.juju-log Gathering list of manually specified brick devices
unit-gluster-0: 20:49:03 DEBUG unit.gluster/0.juju-log Gathering list of manually specified brick devices
unit-gluster-2: 20:49:03 DEBUG unit.gluster/2.juju-log List of manual storage brick devices: ["/dev/sdb"]
unit-gluster-1: 20:49:03 DEBUG unit.gluster/1.juju-log List of manual storage brick devices: ["/dev/sdb"]
unit-gluster-0: 20:49:03 DEBUG unit.gluster/0.juju-log List of manual storage brick devices: ["/dev/sdb"]
unit-gluster-2: 20:49:03 DEBUG unit.gluster/2.juju-log Checking if "/dev/sdb" is a block device
unit-gluster-1: 20:49:03 DEBUG unit.gluster/1.juju-log Checking if "/dev/sdb" is a block device
unit-gluster-0: 20:49:03 DEBUG unit.gluster/0.juju-log Checking if "/dev/sdb" is a block device
unit-gluster-2: 20:49:03 DEBUG unit.gluster/2.juju-log Checking if "/dev/sdb" is initialized
unit-gluster-2: 20:49:03 DEBUG unit.gluster/2.juju-log Connecting to unitdata storage
unit-gluster-0: 20:49:03 DEBUG unit.gluster/0.juju-log Checking if "/dev/sdb" is initialized
unit-gluster-1: 20:49:03 DEBUG unit.gluster/1.juju-log Checking if "/dev/sdb" is initialized
unit-gluster-0: 20:49:03 DEBUG unit.gluster/0.juju-log Connecting to unitdata storage
unit-gluster-1: 20:49:03 DEBUG unit.gluster/1.juju-log Connecting to unitdata storage
unit-gluster-2: 20:49:03 DEBUG unit.gluster/2.juju-log Getting unit_info
unit-gluster-0: 20:49:03 DEBUG unit.gluster/0.juju-log Getting unit_info
unit-gluster-1: 20:49:04 DEBUG unit.gluster/1.juju-log Getting unit_info
unit-gluster-2: 20:49:04 DEBUG unit.gluster/2.juju-log unit_info: None
unit-gluster-0: 20:49:04 DEBUG unit.gluster/0.juju-log unit_info: None
unit-gluster-1: 20:49:04 DEBUG unit.gluster/1.juju-log unit_info: None
unit-gluster-2: 20:49:04 DEBUG unit.gluster/2.juju-log Gathering list of juju storage brick devices
unit-gluster-0: 20:49:04 DEBUG unit.gluster/0.juju-log Gathering list of juju storage brick devices
unit-gluster-1: 20:49:04 DEBUG unit.gluster/1.juju-log Gathering list of juju storage brick devices
unit-gluster-0: 20:49:04 DEBUG unit.gluster/0.juju-log List of juju storage brick devices: ["/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1"]
unit-gluster-2: 20:49:04 DEBUG unit.gluster/2.juju-log List of juju storage brick devices: ["/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1"]
unit-gluster-0: 20:49:04 DEBUG unit.gluster/0.juju-log Checking if "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1" is a block device
unit-gluster-2: 20:49:04 DEBUG unit.gluster/2.juju-log Checking if "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1" is a block device
unit-gluster-1: 20:49:04 DEBUG unit.gluster/1.juju-log List of juju storage brick devices: ["/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1"]
unit-gluster-0: 20:49:04 DEBUG unit.gluster/0.juju-log Skipping invalid block device: "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1"
unit-gluster-1: 20:49:04 DEBUG unit.gluster/1.juju-log Checking if "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1" is a block device
unit-gluster-2: 20:49:04 DEBUG unit.gluster/2.juju-log Skipping invalid block device: "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1"
unit-gluster-0: 20:49:04 DEBUG unit.gluster/0.juju-log storage devices: [BrickDevice { is_block_device: true, initialized: false, mount_path: "/mnt/sdb", dev_path: "/dev/sdb" }]
unit-gluster-2: 20:49:04 DEBUG unit.gluster/2.juju-log storage devices: [BrickDevice { is_block_device: true, initialized: false, mount_path: "/mnt/sdb", dev_path: "/dev/sdb" }]
unit-gluster-1: 20:49:04 DEBUG unit.gluster/1.juju-log Skipping invalid block device: "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1"
unit-gluster-0: 20:49:04 DEBUG unit.gluster/0.juju-log Calling initialize_storage for "/dev/sdb"
unit-gluster-2: 20:49:04 DEBUG unit.gluster/2.juju-log Calling initialize_storage for "/dev/sdb"
unit-gluster-1: 20:49:04 DEBUG unit.gluster/1.juju-log storage devices: [BrickDevice { is_block_device: true, initialized: false, mount_path: "/mnt/sdb", dev_path: "/dev/sdb" }]
unit-gluster-1: 20:49:04 DEBUG unit.gluster/1.juju-log Calling initialize_storage for "/dev/sdb"
unit-gluster-2: 20:49:04 INFO unit.gluster/2.juju-log Formatting block device with Ext4: "/dev/sdb"
unit-gluster-0: 20:49:04 INFO unit.gluster/0.juju-log Formatting block device with Ext4: "/dev/sdb"
unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed mke2fs 1.42.13 (17-May-2015)
unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed Discarding device blocks: done                            
unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed Creating filesystem with 16777216 4k blocks and 4194304 inodes
unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed Filesystem UUID: c648a84c-e273-4c28-bbcc-66a057cda5ca
unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed Superblock backups stored on blocks: 
unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed     32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed     4096000, 7962624, 11239424
unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed 
unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed Allocating group tables: done                            
unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed Writing inode tables: done                            
unit-gluster-1: 20:49:04 INFO unit.gluster/1.juju-log Formatting block device with Ext4: "/dev/sdb"
unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed mke2fs 1.42.13 (17-May-2015)
unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed Discarding device blocks: done                            
unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed Creating filesystem with 16777216 4k blocks and 4194304 inodes
unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed Filesystem UUID: 7c2e7277-a647-4e40-91f9-b2de3e86500d
unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed Superblock backups stored on blocks: 
unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed     32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed     4096000, 7962624, 11239424
unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed 
unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed Allocating group tables: done                            
unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed Writing inode tables: done                            
unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed Creating journal (32768 blocks): done
unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed Creating journal (32768 blocks): done
unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed mke2fs 1.42.13 (17-May-2015)
unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed Discarding device blocks: done                            
unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed Creating filesystem with 16777216 4k blocks and 4194304 inodes
unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed Filesystem UUID: 5a4762f2-ed6e-4dca-8b89-da1012a1cc9c
unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed Superblock backups stored on blocks: 
unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed     32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed     4096000, 7962624, 11239424
unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed 
unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed Allocating group tables: done                            
unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed Writing inode tables: done                            
unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed Creating journal (32768 blocks): done
unit-gluster-1: 20:49:08 INFO unit.gluster/1.config-changed Writing superblocks and filesystem accounting information: done   
unit-gluster-1: 20:49:08 INFO unit.gluster/1.config-changed 
unit-gluster-2: 20:49:08 INFO unit.gluster/2.config-changed Writing superblocks and filesystem accounting information: done   
unit-gluster-2: 20:49:08 INFO unit.gluster/2.config-changed 
unit-gluster-2: 20:49:08 DEBUG unit.gluster/2.juju-log Command output: Output { status: ExitStatus(ExitStatus(0)), stdout: "", stderr: "" }
unit-gluster-1: 20:49:08 DEBUG unit.gluster/1.juju-log Command output: Output { status: ExitStatus(ExitStatus(0)), stdout: "", stderr: "" }
unit-gluster-2: 20:49:08 INFO unit.gluster/2.juju-log device_info: Device { id: Some(Uuid("c648a84c-e273-4c28-bbcc-66a057cda5ca")), name: "sdb", media_type: Unknown, capacity: 68719476736, fs_type: Ext4 }
unit-gluster-2: 20:49:08 INFO unit.gluster/2.juju-log Mounting block device "/dev/sdb" at /mnt/sdb
unit-gluster-1: 20:49:08 INFO unit.gluster/1.juju-log device_info: Device { id: Some(Uuid("5a4762f2-ed6e-4dca-8b89-da1012a1cc9c")), name: "sdb", media_type: Unknown, capacity: 68719476736, fs_type: Ext4 }
unit-gluster-1: 20:49:08 INFO unit.gluster/1.juju-log Mounting block device "/dev/sdb" at /mnt/sdb
unit-gluster-0: 20:49:09 INFO unit.gluster/0.config-changed Writing superblocks and filesystem accounting information: done   
unit-gluster-0: 20:49:09 INFO unit.gluster/0.config-changed 
unit-gluster-0: 20:49:09 DEBUG unit.gluster/0.juju-log Command output: Output { status: ExitStatus(ExitStatus(0)), stdout: "", stderr: "" }
unit-gluster-2: 20:49:09 INFO unit.gluster/2.juju-log Creating mount directory: /mnt/sdb
unit-gluster-1: 20:49:09 INFO unit.gluster/1.juju-log Creating mount directory: /mnt/sdb
unit-gluster-0: 20:49:09 INFO unit.gluster/0.juju-log device_info: Device { id: Some(Uuid("7c2e7277-a647-4e40-91f9-b2de3e86500d")), name: "sdb", media_type: Unknown, capacity: 68719476736, fs_type: Ext4 }
unit-gluster-0: 20:49:09 INFO unit.gluster/0.juju-log Mounting block device "/dev/sdb" at /mnt/sdb
unit-gluster-1: 20:49:09 DEBUG unit.gluster/1.juju-log Command output: Output { status: ExitStatus(ExitStatus(0)), stdout: "", stderr: "" }
unit-gluster-1: 20:49:09 DEBUG unit.gluster/1.juju-log Adding FsEntry { fs_spec: "UUID=5a4762f2-ed6e-4dca-8b89-da1012a1cc9c", mountpoint: "/mnt/sdb", vfs_type: "ext4", mount_options: ["noatime", "inode64"], dump: false, fsck_order: 2 } to fstab
unit-gluster-2: 20:49:09 DEBUG unit.gluster/2.juju-log Command output: Output { status: ExitStatus(ExitStatus(0)), stdout: "", stderr: "" }
unit-gluster-0: 20:49:09 INFO unit.gluster/0.juju-log Creating mount directory: /mnt/sdb
unit-gluster-2: 20:49:09 DEBUG unit.gluster/2.juju-log Adding FsEntry { fs_spec: "UUID=c648a84c-e273-4c28-bbcc-66a057cda5ca", mountpoint: "/mnt/sdb", vfs_type: "ext4", mount_options: ["noatime", "inode64"], dump: false, fsck_order: 2 } to fstab
unit-gluster-1: 20:49:09 INFO unit.gluster/1.juju-log Removing mount path from updatedb "/mnt/sdb"
unit-gluster-1: 20:49:09 DEBUG unit.gluster/1.juju-log Usable brick paths: ["/mnt/sdb"]
unit-gluster-1: 20:49:09 DEBUG unit.gluster/1.juju-log .juju-persistent-config saved.  Wrote 445 bytes
unit-gluster-1: 20:49:09 DEBUG unit.gluster/1.juju-log .juju-persistent-config saved.  Wrote 445 bytes
unit-gluster-0: 20:49:09 DEBUG unit.gluster/0.juju-log Command output: Output { status: ExitStatus(ExitStatus(0)), stdout: "", stderr: "" }
unit-gluster-0: 20:49:09 DEBUG unit.gluster/0.juju-log Adding FsEntry { fs_spec: "UUID=7c2e7277-a647-4e40-91f9-b2de3e86500d", mountpoint: "/mnt/sdb", vfs_type: "ext4", mount_options: ["noatime", "inode64"], dump: false, fsck_order: 2 } to fstab
unit-gluster-2: 20:49:09 INFO unit.gluster/2.juju-log Removing mount path from updatedb "/mnt/sdb"
unit-gluster-2: 20:49:09 DEBUG unit.gluster/2.juju-log Usable brick paths: ["/mnt/sdb"]
unit-gluster-0: 20:49:09 INFO unit.gluster/0.juju-log Removing mount path from updatedb "/mnt/sdb"
unit-gluster-1: 20:49:09 DEBUG unit.gluster/1.juju-log .juju-persistent-config saved.  Wrote 445 bytes
unit-gluster-2: 20:49:09 DEBUG unit.gluster/2.juju-log .juju-persistent-config saved.  Wrote 445 bytes
unit-gluster-0: 20:49:09 DEBUG unit.gluster/0.juju-log Usable brick paths: ["/mnt/sdb"]
unit-gluster-1: 20:49:09 DEBUG unit.gluster/1.juju-log No upgrade requested
unit-gluster-0: 20:49:09 DEBUG unit.gluster/0.juju-log .juju-persistent-config saved.  Wrote 445 bytes
unit-gluster-2: 20:49:09 DEBUG unit.gluster/2.juju-log .juju-persistent-config saved.  Wrote 445 bytes
unit-gluster-0: 20:49:09 DEBUG unit.gluster/0.juju-log .juju-persistent-config saved.  Wrote 445 bytes
unit-gluster-2: 20:49:09 DEBUG unit.gluster/2.juju-log .juju-persistent-config saved.  Wrote 445 bytes
unit-gluster-2: 20:49:09 DEBUG unit.gluster/2.juju-log No upgrade requested
unit-gluster-1: 20:49:09 INFO unit.gluster/1.config-changed Volume info get command failed with error: Volume test does not exist
unit-gluster-1: 20:49:09 INFO unit.gluster/1.config-changed 
unit-gluster-0: 20:49:09 DEBUG unit.gluster/0.juju-log .juju-persistent-config saved.  Wrote 445 bytes
unit-gluster-0: 20:49:09 DEBUG unit.gluster/0.juju-log No upgrade requested
unit-gluster-2: 20:49:10 INFO unit.gluster/2.config-changed Volume info get command failed with error: Volume test does not exist
unit-gluster-2: 20:49:10 INFO unit.gluster/2.config-changed 
unit-gluster-0: 20:49:10 INFO unit.gluster/0.config-changed Volume info get command failed with error: Volume test does not exist
unit-gluster-0: 20:49:10 INFO unit.gluster/0.config-changed 

Same with a clean install with

  brick_devices:
    type: string
    default: /dev/sdb

https://paste.ubuntu.com/24693720/

cholcombe973 commented 7 years ago

That's interesting. I'll see if I can track it down. My first thought was it was getting skipped because it wasn't showing up as a block device but the log messages disagreed. It could also be the filter code that skips block devices it doesn't understand. On Sun, May 28, 2017 at 11:07 AM Dmitrii Shcherbakov < notifications@github.com> wrote:

Tried to do a test on qemu virtual machines with virtio-scsi disks tagged in MAAS.

ubuntu-q87:/tmp$ juju --version 2.2-beta4-xenial-amd64

ubuntu@maas:~$ maas maas version read Success. Machine-readable output follows: {"subversion": "bzr6054-0ubuntu1~16.04.1", "version": "2.2.0", "capabilities": ["networks-management", "static-ipaddresses", "ipv6-deployment-ubuntu", "devices-management", "storage-deployment-ubuntu", "network-deployment-ubuntu", "bridging-interface-ubuntu", "bridging-automatic-ubuntu", "authenticate-api"]}

ubuntu-q87:/tmp$ juju create-storage-pool glusterpool maas tags=jstorage

ubuntu-q87:/tmp$ juju storage-pools Name Provider Attrs glusterpool maas tags=jstorage loop loop maas maas rootfs rootfs tmpfs tmpfs

ubuntu-q87:/tmp$ juju deploy ./gluster-charm -n 3 --storage brick='glusterpool' Deploying charm "local:xenial/gluster-0".

for better error reporting of mk2fs

ubuntu-q87:/tmp$ juju config gluster filesystem_type ext4

ubuntu-q87:/tmp$ juju status Model Controller Cloud/Region Version SLA default vmaas vmaas 2.2-beta4 unsupported

App Version Status Scale Charm Store Rev OS Notes gluster 3.10.2 blocked 3 gluster local 0 ubuntu

Unit Workload Agent Machine Public address Ports Message gluster/0 blocked idle 0 10.10.101.65 No bricks found gluster/1* blocked idle 1 10.10.101.63 No bricks found gluster/2 blocked idle 2 10.10.101.64 No bricks found

Machine State DNS Inst id Series AZ Message 0 started 10.10.101.65 xcgtpc xenial default Deployed 1 started 10.10.101.63 rynftc xenial default Deployed 2 started 10.10.101.64 rw7fq3 xenial default Deployed

Relation Provides Consumes Type

The block device seems to be skipped (see the log below) "Skipping invalid block device: "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1""

ubuntu@maas-xenial5:~$ readlink /dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1 ../../sdb

ubuntu@maas-xenial5:~$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 64G 0 disk └─sda1 8:1 0 64G 0 part / sdb 8:16 0 64G 0 disk

ubuntu@maas-xenial5:~$ lsscsi [2:0:0:0] disk QEMU QEMU HARDDISK 2.5+ /dev/sda [2:0:0:1] disk QEMU QEMU HARDDISK 2.5+ /dev/sdb

unit-gluster-0: 19:31:15 INFO unit.gluster/0.juju-log Checking for new devices unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log Checking for ephemeral unmount unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log Gathering list of manually specified brick devices unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log List of manual storage brick devices: [] unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log Gathering list of juju storage brick devices unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log List of juju storage brick devices: ["/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1"] unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log Checking if "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1" is a block device unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log Skipping invalid block device: "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1" unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log storage devices: [] unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log Usable brick paths: [] unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log .juju-persistent-config saved. Wrote 445 bytes unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log .juju-persistent-config saved. Wrote 445 bytes unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log .juju-persistent-config saved. Wrote 445 bytes unit-gluster-0: 19:31:15 DEBUG unit.gluster/0.juju-log No upgrade requested unit-gluster-0: 19:31:15 INFO unit.gluster/0.config-changed Volume info get command failed with error: Volume test does not exist unit-gluster-0: 19:31:15 INFO unit.gluster/0.config-changed unit-gluster-0: 19:31:16 INFO unit.gluster/0.server-relation-changed Volume info get command failed with error: Volume test does not exist unit-gluster-0: 19:31:16 INFO unit.gluster/0.server-relation-changed unit-gluster-1: 19:31:17 DEBUG unit.gluster/1.juju-log server:0: I am the leader: 0 unit-gluster-1: 19:31:18 INFO unit.gluster/1.juju-log server:0: Loading config unit-gluster-1: 19:31:18 DEBUG unit.gluster/1.juju-log server:0: peer list: [UUID: bc6b036f-2bae-4f2a-8f61-dd04fe1446ae Hostname: 10.10.101.63 Status: Connected] unit-gluster-2: 19:31:18 INFO unit.gluster/2.server-relation-changed Volume info get command failed with error: Volume test does not exist unit-gluster-2: 19:31:18 INFO unit.gluster/2.server-relation-changed unit-gluster-0: 19:31:18 INFO unit.gluster/0.server-relation-changed Volume info get command failed with error: Volume test does not exist unit-gluster-0: 19:31:18 INFO unit.gluster/0.server-relation-changed unit-gluster-1: 19:31:18 DEBUG unit.gluster/1.juju-log server:0: relation-list output: gluster/0

unit-gluster-1: 19:31:18 DEBUG unit.gluster/1.juju-log server:0: Adding in related_units: [Relation { name: "gluster", id: 0 }] unit-gluster-1: 19:31:18 DEBUG unit.gluster/1.juju-log server:0: Adding 10.10.101.65 to cluster unit-gluster-1: 19:31:18 DEBUG unit.gluster/1.juju-log server:0: Gluster peer probe was successful unit-gluster-1: 19:31:18 INFO unit.gluster/1.server-relation-changed Volume info get command failed with error: Volume test does not exist unit-gluster-1: 19:31:18 INFO unit.gluster/1.server-relation-changed unit-gluster-1: 19:31:18 INFO unit.gluster/1.juju-log server:0: Creating volume test unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Waiting for all peers to enter the Peer in Cluster status unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Got peer status: [UUID: c5642b67-77bf-4669-af24-30817c3ff55d Hostname: 10.10.101.65 Status: peer in cluster] unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Checking for ephemeral unmount unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Gathering list of manually specified brick devices unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: List of manual storage brick devices: [] unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Gathering list of juju storage brick devices unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: List of juju storage brick devices: ["/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1"] unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Checking if "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1" is a block device unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Skipping invalid block device: "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1" unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: storage devices: [] unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Usable brick paths: [] unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Volume is none unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Not enough peers to satisfy the replication level for the Gluster volume. Waiting for more peers to join. unit-gluster-1: 19:31:19 INFO unit.gluster/1.juju-log server:0: Waiting for more peers unit-gluster-1: 19:31:19 DEBUG unit.gluster/1.juju-log server:0: Waiting for all peers to enter the Peer in Cluster status unit-gluster-1: 19:31:19 ERROR unit.gluster/1.juju-log server:0: Hook failed with error: Some("Mount failed. Please check the log file for more details.\n") unit-gluster-1: 19:31:19 INFO unit.gluster/1.server-relation-changed Volume info get command failed with error: Volume test does not exist

Adding this device to a list of brick devices without redeploying does not work:

ubuntu-q87:/tmp$ juju config gluster brick_devices=/dev/sdb

unit-gluster-1: 20:49:03 INFO unit.gluster/1.juju-log Checking for new devices unit-gluster-0: 20:49:03 INFO unit.gluster/0.juju-log Checking for new devices unit-gluster-2: 20:49:03 INFO unit.gluster/2.juju-log Checking for new devices unit-gluster-1: 20:49:03 DEBUG unit.gluster/1.juju-log Checking for ephemeral unmount unit-gluster-0: 20:49:03 DEBUG unit.gluster/0.juju-log Checking for ephemeral unmount unit-gluster-2: 20:49:03 DEBUG unit.gluster/2.juju-log Checking for ephemeral unmount unit-gluster-1: 20:49:03 DEBUG unit.gluster/1.juju-log Gathering list of manually specified brick devices unit-gluster-2: 20:49:03 DEBUG unit.gluster/2.juju-log Gathering list of manually specified brick devices unit-gluster-0: 20:49:03 DEBUG unit.gluster/0.juju-log Gathering list of manually specified brick devices unit-gluster-2: 20:49:03 DEBUG unit.gluster/2.juju-log List of manual storage brick devices: ["/dev/sdb"] unit-gluster-1: 20:49:03 DEBUG unit.gluster/1.juju-log List of manual storage brick devices: ["/dev/sdb"] unit-gluster-0: 20:49:03 DEBUG unit.gluster/0.juju-log List of manual storage brick devices: ["/dev/sdb"] unit-gluster-2: 20:49:03 DEBUG unit.gluster/2.juju-log Checking if "/dev/sdb" is a block device unit-gluster-1: 20:49:03 DEBUG unit.gluster/1.juju-log Checking if "/dev/sdb" is a block device unit-gluster-0: 20:49:03 DEBUG unit.gluster/0.juju-log Checking if "/dev/sdb" is a block device unit-gluster-2: 20:49:03 DEBUG unit.gluster/2.juju-log Checking if "/dev/sdb" is initialized unit-gluster-2: 20:49:03 DEBUG unit.gluster/2.juju-log Connecting to unitdata storage unit-gluster-0: 20:49:03 DEBUG unit.gluster/0.juju-log Checking if "/dev/sdb" is initialized unit-gluster-1: 20:49:03 DEBUG unit.gluster/1.juju-log Checking if "/dev/sdb" is initialized unit-gluster-0: 20:49:03 DEBUG unit.gluster/0.juju-log Connecting to unitdata storage unit-gluster-1: 20:49:03 DEBUG unit.gluster/1.juju-log Connecting to unitdata storage unit-gluster-2: 20:49:03 DEBUG unit.gluster/2.juju-log Getting unit_info unit-gluster-0: 20:49:03 DEBUG unit.gluster/0.juju-log Getting unit_info unit-gluster-1: 20:49:04 DEBUG unit.gluster/1.juju-log Getting unit_info unit-gluster-2: 20:49:04 DEBUG unit.gluster/2.juju-log unit_info: None unit-gluster-0: 20:49:04 DEBUG unit.gluster/0.juju-log unit_info: None unit-gluster-1: 20:49:04 DEBUG unit.gluster/1.juju-log unit_info: None unit-gluster-2: 20:49:04 DEBUG unit.gluster/2.juju-log Gathering list of juju storage brick devices unit-gluster-0: 20:49:04 DEBUG unit.gluster/0.juju-log Gathering list of juju storage brick devices unit-gluster-1: 20:49:04 DEBUG unit.gluster/1.juju-log Gathering list of juju storage brick devices unit-gluster-0: 20:49:04 DEBUG unit.gluster/0.juju-log List of juju storage brick devices: ["/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1"] unit-gluster-2: 20:49:04 DEBUG unit.gluster/2.juju-log List of juju storage brick devices: ["/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1"] unit-gluster-0: 20:49:04 DEBUG unit.gluster/0.juju-log Checking if "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1" is a block device unit-gluster-2: 20:49:04 DEBUG unit.gluster/2.juju-log Checking if "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1" is a block device unit-gluster-1: 20:49:04 DEBUG unit.gluster/1.juju-log List of juju storage brick devices: ["/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1"] unit-gluster-0: 20:49:04 DEBUG unit.gluster/0.juju-log Skipping invalid block device: "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1" unit-gluster-1: 20:49:04 DEBUG unit.gluster/1.juju-log Checking if "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1" is a block device unit-gluster-2: 20:49:04 DEBUG unit.gluster/2.juju-log Skipping invalid block device: "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1" unit-gluster-0: 20:49:04 DEBUG unit.gluster/0.juju-log storage devices: [BrickDevice { is_block_device: true, initialized: false, mount_path: "/mnt/sdb", dev_path: "/dev/sdb" }] unit-gluster-2: 20:49:04 DEBUG unit.gluster/2.juju-log storage devices: [BrickDevice { is_block_device: true, initialized: false, mount_path: "/mnt/sdb", dev_path: "/dev/sdb" }] unit-gluster-1: 20:49:04 DEBUG unit.gluster/1.juju-log Skipping invalid block device: "/dev/disk/by-id/scsi-0QEMU_QEMU_HARDDISK_drive-scsi0-0-0-1" unit-gluster-0: 20:49:04 DEBUG unit.gluster/0.juju-log Calling initialize_storage for "/dev/sdb" unit-gluster-2: 20:49:04 DEBUG unit.gluster/2.juju-log Calling initialize_storage for "/dev/sdb" unit-gluster-1: 20:49:04 DEBUG unit.gluster/1.juju-log storage devices: [BrickDevice { is_block_device: true, initialized: false, mount_path: "/mnt/sdb", dev_path: "/dev/sdb" }] unit-gluster-1: 20:49:04 DEBUG unit.gluster/1.juju-log Calling initialize_storage for "/dev/sdb" unit-gluster-2: 20:49:04 INFO unit.gluster/2.juju-log Formatting block device with Ext4: "/dev/sdb" unit-gluster-0: 20:49:04 INFO unit.gluster/0.juju-log Formatting block device with Ext4: "/dev/sdb" unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed mke2fs 1.42.13 (17-May-2015) unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed Discarding device blocks: done unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed Creating filesystem with 16777216 4k blocks and 4194304 inodes unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed Filesystem UUID: c648a84c-e273-4c28-bbcc-66a057cda5ca unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed Superblock backups stored on blocks: unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed 4096000, 7962624, 11239424 unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed Allocating group tables: done unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed Writing inode tables: done unit-gluster-1: 20:49:04 INFO unit.gluster/1.juju-log Formatting block device with Ext4: "/dev/sdb" unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed mke2fs 1.42.13 (17-May-2015) unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed Discarding device blocks: done unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed Creating filesystem with 16777216 4k blocks and 4194304 inodes unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed Filesystem UUID: 7c2e7277-a647-4e40-91f9-b2de3e86500d unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed Superblock backups stored on blocks: unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed 4096000, 7962624, 11239424 unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed Allocating group tables: done unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed Writing inode tables: done unit-gluster-2: 20:49:04 INFO unit.gluster/2.config-changed Creating journal (32768 blocks): done unit-gluster-0: 20:49:04 INFO unit.gluster/0.config-changed Creating journal (32768 blocks): done unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed mke2fs 1.42.13 (17-May-2015) unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed Discarding device blocks: done unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed Creating filesystem with 16777216 4k blocks and 4194304 inodes unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed Filesystem UUID: 5a4762f2-ed6e-4dca-8b89-da1012a1cc9c unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed Superblock backups stored on blocks: unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed 4096000, 7962624, 11239424 unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed Allocating group tables: done unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed Writing inode tables: done unit-gluster-1: 20:49:04 INFO unit.gluster/1.config-changed Creating journal (32768 blocks): done unit-gluster-1: 20:49:08 INFO unit.gluster/1.config-changed Writing superblocks and filesystem accounting information: done unit-gluster-1: 20:49:08 INFO unit.gluster/1.config-changed unit-gluster-2: 20:49:08 INFO unit.gluster/2.config-changed Writing superblocks and filesystem accounting information: done unit-gluster-2: 20:49:08 INFO unit.gluster/2.config-changed unit-gluster-2: 20:49:08 DEBUG unit.gluster/2.juju-log Command output: Output { status: ExitStatus(ExitStatus(0)), stdout: "", stderr: "" } unit-gluster-1: 20:49:08 DEBUG unit.gluster/1.juju-log Command output: Output { status: ExitStatus(ExitStatus(0)), stdout: "", stderr: "" } unit-gluster-2: 20:49:08 INFO unit.gluster/2.juju-log device_info: Device { id: Some(Uuid("c648a84c-e273-4c28-bbcc-66a057cda5ca")), name: "sdb", media_type: Unknown, capacity: 68719476736, fs_type: Ext4 } unit-gluster-2: 20:49:08 INFO unit.gluster/2.juju-log Mounting block device "/dev/sdb" at /mnt/sdb unit-gluster-1: 20:49:08 INFO unit.gluster/1.juju-log device_info: Device { id: Some(Uuid("5a4762f2-ed6e-4dca-8b89-da1012a1cc9c")), name: "sdb", media_type: Unknown, capacity: 68719476736, fs_type: Ext4 } unit-gluster-1: 20:49:08 INFO unit.gluster/1.juju-log Mounting block device "/dev/sdb" at /mnt/sdb unit-gluster-0: 20:49:09 INFO unit.gluster/0.config-changed Writing superblocks and filesystem accounting information: done unit-gluster-0: 20:49:09 INFO unit.gluster/0.config-changed unit-gluster-0: 20:49:09 DEBUG unit.gluster/0.juju-log Command output: Output { status: ExitStatus(ExitStatus(0)), stdout: "", stderr: "" } unit-gluster-2: 20:49:09 INFO unit.gluster/2.juju-log Creating mount directory: /mnt/sdb unit-gluster-1: 20:49:09 INFO unit.gluster/1.juju-log Creating mount directory: /mnt/sdb unit-gluster-0: 20:49:09 INFO unit.gluster/0.juju-log device_info: Device { id: Some(Uuid("7c2e7277-a647-4e40-91f9-b2de3e86500d")), name: "sdb", media_type: Unknown, capacity: 68719476736, fs_type: Ext4 } unit-gluster-0: 20:49:09 INFO unit.gluster/0.juju-log Mounting block device "/dev/sdb" at /mnt/sdb unit-gluster-1: 20:49:09 DEBUG unit.gluster/1.juju-log Command output: Output { status: ExitStatus(ExitStatus(0)), stdout: "", stderr: "" } unit-gluster-1: 20:49:09 DEBUG unit.gluster/1.juju-log Adding FsEntry { fs_spec: "UUID=5a4762f2-ed6e-4dca-8b89-da1012a1cc9c", mountpoint: "/mnt/sdb", vfs_type: "ext4", mount_options: ["noatime", "inode64"], dump: false, fsck_order: 2 } to fstab unit-gluster-2: 20:49:09 DEBUG unit.gluster/2.juju-log Command output: Output { status: ExitStatus(ExitStatus(0)), stdout: "", stderr: "" } unit-gluster-0: 20:49:09 INFO unit.gluster/0.juju-log Creating mount directory: /mnt/sdb unit-gluster-2: 20:49:09 DEBUG unit.gluster/2.juju-log Adding FsEntry { fs_spec: "UUID=c648a84c-e273-4c28-bbcc-66a057cda5ca", mountpoint: "/mnt/sdb", vfs_type: "ext4", mount_options: ["noatime", "inode64"], dump: false, fsck_order: 2 } to fstab unit-gluster-1: 20:49:09 INFO unit.gluster/1.juju-log Removing mount path from updatedb "/mnt/sdb" unit-gluster-1: 20:49:09 DEBUG unit.gluster/1.juju-log Usable brick paths: ["/mnt/sdb"] unit-gluster-1: 20:49:09 DEBUG unit.gluster/1.juju-log .juju-persistent-config saved. Wrote 445 bytes unit-gluster-1: 20:49:09 DEBUG unit.gluster/1.juju-log .juju-persistent-config saved. Wrote 445 bytes unit-gluster-0: 20:49:09 DEBUG unit.gluster/0.juju-log Command output: Output { status: ExitStatus(ExitStatus(0)), stdout: "", stderr: "" } unit-gluster-0: 20:49:09 DEBUG unit.gluster/0.juju-log Adding FsEntry { fs_spec: "UUID=7c2e7277-a647-4e40-91f9-b2de3e86500d", mountpoint: "/mnt/sdb", vfs_type: "ext4", mount_options: ["noatime", "inode64"], dump: false, fsck_order: 2 } to fstab unit-gluster-2: 20:49:09 INFO unit.gluster/2.juju-log Removing mount path from updatedb "/mnt/sdb" unit-gluster-2: 20:49:09 DEBUG unit.gluster/2.juju-log Usable brick paths: ["/mnt/sdb"] unit-gluster-0: 20:49:09 INFO unit.gluster/0.juju-log Removing mount path from updatedb "/mnt/sdb" unit-gluster-1: 20:49:09 DEBUG unit.gluster/1.juju-log .juju-persistent-config saved. Wrote 445 bytes unit-gluster-2: 20:49:09 DEBUG unit.gluster/2.juju-log .juju-persistent-config saved. Wrote 445 bytes unit-gluster-0: 20:49:09 DEBUG unit.gluster/0.juju-log Usable brick paths: ["/mnt/sdb"] unit-gluster-1: 20:49:09 DEBUG unit.gluster/1.juju-log No upgrade requested unit-gluster-0: 20:49:09 DEBUG unit.gluster/0.juju-log .juju-persistent-config saved. Wrote 445 bytes unit-gluster-2: 20:49:09 DEBUG unit.gluster/2.juju-log .juju-persistent-config saved. Wrote 445 bytes unit-gluster-0: 20:49:09 DEBUG unit.gluster/0.juju-log .juju-persistent-config saved. Wrote 445 bytes unit-gluster-2: 20:49:09 DEBUG unit.gluster/2.juju-log .juju-persistent-config saved. Wrote 445 bytes unit-gluster-2: 20:49:09 DEBUG unit.gluster/2.juju-log No upgrade requested unit-gluster-1: 20:49:09 INFO unit.gluster/1.config-changed Volume info get command failed with error: Volume test does not exist unit-gluster-1: 20:49:09 INFO unit.gluster/1.config-changed unit-gluster-0: 20:49:09 DEBUG unit.gluster/0.juju-log .juju-persistent-config saved. Wrote 445 bytes unit-gluster-0: 20:49:09 DEBUG unit.gluster/0.juju-log No upgrade requested unit-gluster-2: 20:49:10 INFO unit.gluster/2.config-changed Volume info get command failed with error: Volume test does not exist unit-gluster-2: 20:49:10 INFO unit.gluster/2.config-changed unit-gluster-0: 20:49:10 INFO unit.gluster/0.config-changed Volume info get command failed with error: Volume test does not exist unit-gluster-0: 20:49:10 INFO unit.gluster/0.config-changed

Same with a clean install with

brick_devices: type: string default: /dev/sdb

https://paste.ubuntu.com/24693720/

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/cholcombe973/gluster-charm/issues/31, or mute the thread https://github.com/notifications/unsubscribe-auth/AC6qEy4pcQLAza-CCfTxCCxd8hZYIqn9ks5r-bfXgaJpZM4Notuu .

cholcombe973 commented 7 years ago

So it looks like the same disk is added twice. One for juju storage and then again manually. @dshcherb does the new patch to detect virtual disks help? In your paste it seems that the leader thinks the disk isn't usable but the others use it. The leader and followers take slightly different paths to initialize storage I'll see if I can track down what it going on. I tried to make the paths as close to identical as possible. They should all either agree the disk is useable or skip it

dshcherb commented 7 years ago

The second log was collected after configuring brick_devices without removing juju storage.

The patch helped with only manual configuration via brick_devices - so that worked well.

I'll paste a log collected with a patch with Juju storage soon.