search

stratis-storage / stratisd

Easy to use local storage management for Linux.
https://stratis-storage.github.io
Mozilla Public License 2.0
789 stars 56 forks source link

Filesystem limited to 1 TB? need more, howto? #1456

Closed ingothierack closed 5 years ago

ingothierack commented 5 years ago

I created a few filesystems out of a pool which works so far. Now I need the first on to be grow beyond 1 TB size. It doesn't do it automatically, so how to grow?

Pool + current usage

Name              Total Physical Size  Total Physical Used
stratis_pool01               7.28 TiB             3.95 TiB

Filesystem with 1 TB stratis_pool01 fs_share 1 TiB Mar 03 2019 02:11 /stratis/stratis_pool01/fs_share a27ab6d2c3634d5ebe73f141d6d051c1

Filesystem full :( /dev/mapper/stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-a27ab6d2c3634d5ebe73f141d6d051c1 1073217536 1073217472 64 100% /mnt/share-new

need here

If there must be done something manually, I didn't find any documentation about it.

Stratis is 1.0.2 on a fedora 29 server installation.

ingothierack commented 5 years ago

Strange behavior. Its now extended to 2 TB, but don't know exactly when this happens. Definitely not when the filesytem was filled up. The extension happens a few hours after it was filled up. Is there some explanation, when the extending will happen?

trgill commented 5 years ago

The filesystem should have grown automatically when the utilization of the filesystem approached the initial 1TB size for XFS. This is a bug.

How are you adding data to the filesystem?

What type of storage are you using? (SSD, NVMe, Rotating)

Just looking for ideas to help reproduce the issue.

ingothierack commented 5 years ago

I did an copy from another disk(mdraid/lvm based) via cp direct disk to disk. In the first run, it fills up the disk, and than stop due to disk full. The source is around 1.3 TB stopped at 1 TB. The disk is a mdraid 5 with 3x 4 TB rotating disks for the stratis pool. I use this system as NAS to provide space via samba and nfs.

What i also see if i copy the the stratis filesystems, is for following log entries. What i'm curios is about the WARN requestet 32768 sectors but extended by 0

Mar 3 10:08:24 nas001 stratisd[751]: INFO libstratis::engine::strat_engine::thinpool::thinpool: Attempting to extend thinpool metadata sub-device belonging to pool 210eb8dac6cb440993454dac6a885dae by 3 2768 sectors Mar 3 10:08:24 nas001 stratisd[751]: WARN libstratis::engine::strat_engine::thinpool::thinpool: Insufficient free space available in backstore; extended thinpool metadata sub-device belonging to pool w ith uuid 210eb8dac6cb440993454dac6a885dae by 0 sectors, request was 32768 sectors

trgill commented 5 years ago

Could you please attach the output from the following commands?

stratis pool list stratis filesystem list lsblk

ingothierack commented 5 years ago 
Name              Total Physical Size  Total Physical Used
stratis_pool01               7.28 TiB             4.41 TiB

I miss in the stratis filesystem list the "problem" filesystem fs_share its mounted as /mnt/share-new /dev/mapper/stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-a27ab6d2c3634d5ebe73f141d6d051c1 2146959360 1299280444 847678916 61% /mnt/share-new

Pool Name       Name             Used         Created            Device                                   UUID                              
stratis_pool01  fs_picture       286.05 GiB   Dec 30 2018 11:10  /stratis/stratis_pool01/fs_picture       f9bd986f41734a879adec71fb4304453  
stratis_pool01  fs_book          130.35 GiB   Dec 22 2018 16:58  /stratis/stratis_pool01/fs_book          e9e709b92e9c4960ba247a5b3163a427  
stratis_pool01  fs_git           1.62 GiB     Mar 05 2019 20:42  /stratis/stratis_pool01/fs_git           4de11f6852de449ba59a99580d98382e  
stratis_pool01  fs_media         390.36 GiB   Dec 22 2018 16:59  /stratis/stratis_pool01/fs_media         33e155ee6ffe4e96a04f2ca517827491  
stratis_pool01  fs_archiv        564.36 GiB   Mar 01 2019 13:10  /stratis/stratis_pool01/fs_archiv        45fa24827f264f24a61028fbb1d3824a  
stratis_pool01  fs_download      1023.95 GiB  Dec 22 2018 21:54  /stratis/stratis_pool01/fs_download      55e0f6b9c7e04cd499a34363b452d71a  
stratis_pool01  fs_remotebackup  50.06 GiB    Dec 31 2018 14:24  /stratis/stratis_pool01/fs_remotebackup  8272dc98f3b84577a1197e77123ee116  
stratis_pool01  fs_mirror        235.36 GiB   Mar 05 2019 21:04  /stratis/stratis_pool01/fs_mirror        ef52c734ca1a48739951efd49963924f  
stratis_pool01  fs_torrent       190.71 GiB   Dec 22 2018 16:56  /stratis/stratis_pool01/fs_torrent       d9e69bf07fd44f23900b529d0e4c8d6b  
stratis_pool01  fs_backup        138.59 GiB   Mar 02 2019 17:32  /stratis/stratis_pool01/fs_backup        726518f251c74b3b944207c940039014  
stratis_pool01  fs_dockerrepo    581 MiB      Jan 13 2019 09:48  /stratis/stratis_pool01/fs_dockerrepo    b9fda4980f644206bb4869776fed9201  
stratis_pool01  fs_home          138.35 GiB   Mar 02 2019 16:58  /stratis/stratis_pool01/fs_home          ea926f46eb6f41b08b51c9442adb5ce3  
stratis_pool01  fs_bucketstore   1.88 GiB     Jan 27 2019 15:42  /stratis/stratis_pool01/fs_bucketstore   e0b7d6e183894cfcbefee56ad3c29b64  
stratis_pool01  fs_workarea      97.24 GiB    Mar 01 2019 08:10  /stratis/stratis_pool01/fs_workarea      70f29e65dc194bbfb705c4987534b2ff

the md127 is the "old" storage lvm based, md126 the new one with stratis

NAME                                                                                            MAJ:MIN RM   SIZE RO TYPE    MOUNTPOINT
sda                                                                                               8:0    1 238.5G  0 disk    
├─sda1                                                                                            8:1    1   200M  0 part    /boot/efi
├─sda2                                                                                            8:2    1     1G  0 part    /boot
└─sda3                                                                                            8:3    1 237.3G  0 part    
  ├─fedora-root                                                                                 253:0    0    15G  0 lvm     /
  └─fedora-swap                                                                                 253:1    0   7.9G  0 lvm     [SWAP]
sdb                                                                                               8:16   0   1.8T  0 disk    
└─md127                                                                                           9:127  0   7.3T  0 raid5   
  ├─vg--master--vm1--disk01-lv--disk01--picture                                                 253:18   0  1000G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--home                                                    253:19   0   300G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--share                                                   253:20   0   1.5T  0 lvm     /mnt/share
  ├─vg--master--vm1--disk01-lv--disk01--newsgroup                                               253:21   0    50G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--torrent                                                 253:22   0   150G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--archiv                                                  253:23   0   600G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--mirror                                                  253:24   0   250G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--backup                                                  253:25   0  1000G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--media                                                   253:26   0   400G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--workarea                                                253:27   0   150G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--book                                                    253:28   0   200G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--sync                                                    253:29   0    50G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--stage                                                   253:30   0   150G  0 lvm     /mnt/stage
  ├─vg--master--vm1--disk01-lv--disk01--git                                                     253:31   0    50G  0 lvm     
  └─vg--master--vm1--disk01-lv--disk01--download                                                253:32   0  1000G  0 lvm     
sdc                                                                                               8:32   0   1.8T  0 disk    
└─md127                                                                                           9:127  0   7.3T  0 raid5   
  ├─vg--master--vm1--disk01-lv--disk01--picture                                                 253:18   0  1000G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--home                                                    253:19   0   300G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--share                                                   253:20   0   1.5T  0 lvm     /mnt/share
  ├─vg--master--vm1--disk01-lv--disk01--newsgroup                                               253:21   0    50G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--torrent                                                 253:22   0   150G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--archiv                                                  253:23   0   600G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--mirror                                                  253:24   0   250G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--backup                                                  253:25   0  1000G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--media                                                   253:26   0   400G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--workarea                                                253:27   0   150G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--book                                                    253:28   0   200G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--sync                                                    253:29   0    50G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--stage                                                   253:30   0   150G  0 lvm     /mnt/stage
  ├─vg--master--vm1--disk01-lv--disk01--git                                                     253:31   0    50G  0 lvm     
  └─vg--master--vm1--disk01-lv--disk01--download                                                253:32   0  1000G  0 lvm     
sdd                                                                                               8:48   0   1.8T  0 disk    
└─md127                                                                                           9:127  0   7.3T  0 raid5   
  ├─vg--master--vm1--disk01-lv--disk01--picture                                                 253:18   0  1000G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--home                                                    253:19   0   300G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--share                                                   253:20   0   1.5T  0 lvm     /mnt/share
  ├─vg--master--vm1--disk01-lv--disk01--newsgroup                                               253:21   0    50G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--torrent                                                 253:22   0   150G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--archiv                                                  253:23   0   600G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--mirror                                                  253:24   0   250G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--backup                                                  253:25   0  1000G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--media                                                   253:26   0   400G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--workarea                                                253:27   0   150G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--book                                                    253:28   0   200G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--sync                                                    253:29   0    50G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--stage                                                   253:30   0   150G  0 lvm     /mnt/stage
  ├─vg--master--vm1--disk01-lv--disk01--git                                                     253:31   0    50G  0 lvm     
  └─vg--master--vm1--disk01-lv--disk01--download                                                253:32   0  1000G  0 lvm     
sde                                                                                               8:64   0   1.8T  0 disk    
└─md127                                                                                           9:127  0   7.3T  0 raid5   
  ├─vg--master--vm1--disk01-lv--disk01--picture                                                 253:18   0  1000G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--home                                                    253:19   0   300G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--share                                                   253:20   0   1.5T  0 lvm     /mnt/share
  ├─vg--master--vm1--disk01-lv--disk01--newsgroup                                               253:21   0    50G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--torrent                                                 253:22   0   150G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--archiv                                                  253:23   0   600G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--mirror                                                  253:24   0   250G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--backup                                                  253:25   0  1000G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--media                                                   253:26   0   400G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--workarea                                                253:27   0   150G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--book                                                    253:28   0   200G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--sync                                                    253:29   0    50G  0 lvm     
  ├─vg--master--vm1--disk01-lv--disk01--stage                                                   253:30   0   150G  0 lvm     /mnt/stage
  ├─vg--master--vm1--disk01-lv--disk01--git                                                     253:31   0    50G  0 lvm     
  └─vg--master--vm1--disk01-lv--disk01--download                                                253:32   0  1000G  0 lvm     
sdf                                                                                               8:80   0   1.8T  0 disk    
sdg                                                                                               8:96   0   3.7T  0 disk    
└─sdg1                                                                                            8:97   0   3.7T  0 part    
  └─md126                                                                                         9:126  0   7.3T  0 raid5   
    └─stratis-1-private-210eb8dac6cb440993454dac6a885dae-physical-originsub                     253:2    0   7.3T  0 stratis 
      ├─stratis-1-private-210eb8dac6cb440993454dac6a885dae-flex-thinmeta                        253:3    0   7.4G  0 stratis 
      │ └─stratis-1-private-210eb8dac6cb440993454dac6a885dae-thinpool-pool                      253:5    0   7.3T  0 stratis 
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-d9e69bf07fd44f23900b529d0e4c8d6b 253:7    0     1T  0 stratis /mnt/torrent
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-e9e709b92e9c4960ba247a5b3163a427 253:8    0     1T  0 stratis /mnt/book
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-33e155ee6ffe4e96a04f2ca517827491 253:9    0     1T  0 stratis /mnt/media
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-55e0f6b9c7e04cd499a34363b452d71a 253:10   0     1T  0 stratis /mnt/download-new
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-f9bd986f41734a879adec71fb4304453 253:11   0     1T  0 stratis /mnt/picture-new
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-8272dc98f3b84577a1197e77123ee116 253:12   0     1T  0 stratis /mnt/remotebackup
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-b9fda4980f644206bb4869776fed9201 253:13   0     1T  0 stratis /mnt/dockerrepo
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-e0b7d6e183894cfcbefee56ad3c29b64 253:14   0     1T  0 stratis /mnt/bucketstore
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-70f29e65dc194bbfb705c4987534b2ff 253:15   0     1T  0 stratis /mnt/workarea
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-45fa24827f264f24a61028fbb1d3824a 253:16   0     1T  0 stratis /mnt/archiv
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-ea926f46eb6f41b08b51c9442adb5ce3 253:17   0     1T  0 stratis /mnt/home
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-726518f251c74b3b944207c940039014 253:33   0     1T  0 stratis /mnt/backup
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-a27ab6d2c3634d5ebe73f141d6d051c1 253:34   0     2T  0 stratis /mnt/share-new
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-4de11f6852de449ba59a99580d98382e 253:35   0     1T  0 stratis /mnt/git
      │   └─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-ef52c734ca1a48739951efd49963924f 253:36   0     1T  0 stratis /mnt/mirror
      ├─stratis-1-private-210eb8dac6cb440993454dac6a885dae-flex-thindata                        253:4    0   7.3T  0 stratis 
      │ └─stratis-1-private-210eb8dac6cb440993454dac6a885dae-thinpool-pool                      253:5    0   7.3T  0 stratis 
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-d9e69bf07fd44f23900b529d0e4c8d6b 253:7    0     1T  0 stratis /mnt/torrent
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-e9e709b92e9c4960ba247a5b3163a427 253:8    0     1T  0 stratis /mnt/book
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-33e155ee6ffe4e96a04f2ca517827491 253:9    0     1T  0 stratis /mnt/media
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-55e0f6b9c7e04cd499a34363b452d71a 253:10   0     1T  0 stratis /mnt/download-new
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-f9bd986f41734a879adec71fb4304453 253:11   0     1T  0 stratis /mnt/picture-new
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-8272dc98f3b84577a1197e77123ee116 253:12   0     1T  0 stratis /mnt/remotebackup
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-b9fda4980f644206bb4869776fed9201 253:13   0     1T  0 stratis /mnt/dockerrepo
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-e0b7d6e183894cfcbefee56ad3c29b64 253:14   0     1T  0 stratis /mnt/bucketstore
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-70f29e65dc194bbfb705c4987534b2ff 253:15   0     1T  0 stratis /mnt/workarea
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-45fa24827f264f24a61028fbb1d3824a 253:16   0     1T  0 stratis /mnt/archiv
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-ea926f46eb6f41b08b51c9442adb5ce3 253:17   0     1T  0 stratis /mnt/home
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-726518f251c74b3b944207c940039014 253:33   0     1T  0 stratis /mnt/backup
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-a27ab6d2c3634d5ebe73f141d6d051c1 253:34   0     2T  0 stratis /mnt/share-new
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-4de11f6852de449ba59a99580d98382e 253:35   0     1T  0 stratis /mnt/git
      │   └─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-ef52c734ca1a48739951efd49963924f 253:36   0     1T  0 stratis /mnt/mirror
      └─stratis-1-private-210eb8dac6cb440993454dac6a885dae-flex-mdv                             253:6    0    16M  0 stratis 
sdh                                                                                               8:112  0   3.7T  0 disk    
└─sdh1                                                                                            8:113  0   3.7T  0 part    
  └─md126                                                                                         9:126  0   7.3T  0 raid5   
    └─stratis-1-private-210eb8dac6cb440993454dac6a885dae-physical-originsub                     253:2    0   7.3T  0 stratis 
      ├─stratis-1-private-210eb8dac6cb440993454dac6a885dae-flex-thinmeta                        253:3    0   7.4G  0 stratis 
      │ └─stratis-1-private-210eb8dac6cb440993454dac6a885dae-thinpool-pool                      253:5    0   7.3T  0 stratis 
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-d9e69bf07fd44f23900b529d0e4c8d6b 253:7    0     1T  0 stratis /mnt/torrent
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-e9e709b92e9c4960ba247a5b3163a427 253:8    0     1T  0 stratis /mnt/book
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-33e155ee6ffe4e96a04f2ca517827491 253:9    0     1T  0 stratis /mnt/media
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-55e0f6b9c7e04cd499a34363b452d71a 253:10   0     1T  0 stratis /mnt/download-new
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-f9bd986f41734a879adec71fb4304453 253:11   0     1T  0 stratis /mnt/picture-new
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-8272dc98f3b84577a1197e77123ee116 253:12   0     1T  0 stratis /mnt/remotebackup
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-b9fda4980f644206bb4869776fed9201 253:13   0     1T  0 stratis /mnt/dockerrepo
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-e0b7d6e183894cfcbefee56ad3c29b64 253:14   0     1T  0 stratis /mnt/bucketstore
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-70f29e65dc194bbfb705c4987534b2ff 253:15   0     1T  0 stratis /mnt/workarea
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-45fa24827f264f24a61028fbb1d3824a 253:16   0     1T  0 stratis /mnt/archiv
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-ea926f46eb6f41b08b51c9442adb5ce3 253:17   0     1T  0 stratis /mnt/home
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-726518f251c74b3b944207c940039014 253:33   0     1T  0 stratis /mnt/backup
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-a27ab6d2c3634d5ebe73f141d6d051c1 253:34   0     2T  0 stratis /mnt/share-new
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-4de11f6852de449ba59a99580d98382e 253:35   0     1T  0 stratis /mnt/git
      │   └─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-ef52c734ca1a48739951efd49963924f 253:36   0     1T  0 stratis /mnt/mirror
      ├─stratis-1-private-210eb8dac6cb440993454dac6a885dae-flex-thindata                        253:4    0   7.3T  0 stratis 
      │ └─stratis-1-private-210eb8dac6cb440993454dac6a885dae-thinpool-pool                      253:5    0   7.3T  0 stratis 
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-d9e69bf07fd44f23900b529d0e4c8d6b 253:7    0     1T  0 stratis /mnt/torrent
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-e9e709b92e9c4960ba247a5b3163a427 253:8    0     1T  0 stratis /mnt/book
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-33e155ee6ffe4e96a04f2ca517827491 253:9    0     1T  0 stratis /mnt/media
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-55e0f6b9c7e04cd499a34363b452d71a 253:10   0     1T  0 stratis /mnt/download-new
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-f9bd986f41734a879adec71fb4304453 253:11   0     1T  0 stratis /mnt/picture-new
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-8272dc98f3b84577a1197e77123ee116 253:12   0     1T  0 stratis /mnt/remotebackup
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-b9fda4980f644206bb4869776fed9201 253:13   0     1T  0 stratis /mnt/dockerrepo
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-e0b7d6e183894cfcbefee56ad3c29b64 253:14   0     1T  0 stratis /mnt/bucketstore
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-70f29e65dc194bbfb705c4987534b2ff 253:15   0     1T  0 stratis /mnt/workarea
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-45fa24827f264f24a61028fbb1d3824a 253:16   0     1T  0 stratis /mnt/archiv
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-ea926f46eb6f41b08b51c9442adb5ce3 253:17   0     1T  0 stratis /mnt/home
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-726518f251c74b3b944207c940039014 253:33   0     1T  0 stratis /mnt/backup
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-a27ab6d2c3634d5ebe73f141d6d051c1 253:34   0     2T  0 stratis /mnt/share-new
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-4de11f6852de449ba59a99580d98382e 253:35   0     1T  0 stratis /mnt/git
      │   └─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-ef52c734ca1a48739951efd49963924f 253:36   0     1T  0 stratis /mnt/mirror
      └─stratis-1-private-210eb8dac6cb440993454dac6a885dae-flex-mdv                             253:6    0    16M  0 stratis 
sdi                                                                                               8:128  0   3.7T  0 disk    
└─sdi1                                                                                            8:129  0   3.7T  0 part    
  └─md126                                                                                         9:126  0   7.3T  0 raid5   
    └─stratis-1-private-210eb8dac6cb440993454dac6a885dae-physical-originsub                     253:2    0   7.3T  0 stratis 
      ├─stratis-1-private-210eb8dac6cb440993454dac6a885dae-flex-thinmeta                        253:3    0   7.4G  0 stratis 
      │ └─stratis-1-private-210eb8dac6cb440993454dac6a885dae-thinpool-pool                      253:5    0   7.3T  0 stratis 
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-d9e69bf07fd44f23900b529d0e4c8d6b 253:7    0     1T  0 stratis /mnt/torrent
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-e9e709b92e9c4960ba247a5b3163a427 253:8    0     1T  0 stratis /mnt/book
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-33e155ee6ffe4e96a04f2ca517827491 253:9    0     1T  0 stratis /mnt/media
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-55e0f6b9c7e04cd499a34363b452d71a 253:10   0     1T  0 stratis /mnt/download-new
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-f9bd986f41734a879adec71fb4304453 253:11   0     1T  0 stratis /mnt/picture-new
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-8272dc98f3b84577a1197e77123ee116 253:12   0     1T  0 stratis /mnt/remotebackup
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-b9fda4980f644206bb4869776fed9201 253:13   0     1T  0 stratis /mnt/dockerrepo
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-e0b7d6e183894cfcbefee56ad3c29b64 253:14   0     1T  0 stratis /mnt/bucketstore
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-70f29e65dc194bbfb705c4987534b2ff 253:15   0     1T  0 stratis /mnt/workarea
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-45fa24827f264f24a61028fbb1d3824a 253:16   0     1T  0 stratis /mnt/archiv
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-ea926f46eb6f41b08b51c9442adb5ce3 253:17   0     1T  0 stratis /mnt/home
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-726518f251c74b3b944207c940039014 253:33   0     1T  0 stratis /mnt/backup
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-a27ab6d2c3634d5ebe73f141d6d051c1 253:34   0     2T  0 stratis /mnt/share-new
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-4de11f6852de449ba59a99580d98382e 253:35   0     1T  0 stratis /mnt/git
      │   └─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-ef52c734ca1a48739951efd49963924f 253:36   0     1T  0 stratis /mnt/mirror
      ├─stratis-1-private-210eb8dac6cb440993454dac6a885dae-flex-thindata                        253:4    0   7.3T  0 stratis 
      │ └─stratis-1-private-210eb8dac6cb440993454dac6a885dae-thinpool-pool                      253:5    0   7.3T  0 stratis 
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-d9e69bf07fd44f23900b529d0e4c8d6b 253:7    0     1T  0 stratis /mnt/torrent
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-e9e709b92e9c4960ba247a5b3163a427 253:8    0     1T  0 stratis /mnt/book
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-33e155ee6ffe4e96a04f2ca517827491 253:9    0     1T  0 stratis /mnt/media
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-55e0f6b9c7e04cd499a34363b452d71a 253:10   0     1T  0 stratis /mnt/download-new
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-f9bd986f41734a879adec71fb4304453 253:11   0     1T  0 stratis /mnt/picture-new
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-8272dc98f3b84577a1197e77123ee116 253:12   0     1T  0 stratis /mnt/remotebackup
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-b9fda4980f644206bb4869776fed9201 253:13   0     1T  0 stratis /mnt/dockerrepo
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-e0b7d6e183894cfcbefee56ad3c29b64 253:14   0     1T  0 stratis /mnt/bucketstore
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-70f29e65dc194bbfb705c4987534b2ff 253:15   0     1T  0 stratis /mnt/workarea
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-45fa24827f264f24a61028fbb1d3824a 253:16   0     1T  0 stratis /mnt/archiv
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-ea926f46eb6f41b08b51c9442adb5ce3 253:17   0     1T  0 stratis /mnt/home
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-726518f251c74b3b944207c940039014 253:33   0     1T  0 stratis /mnt/backup
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-a27ab6d2c3634d5ebe73f141d6d051c1 253:34   0     2T  0 stratis /mnt/share-new
      │   ├─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-4de11f6852de449ba59a99580d98382e 253:35   0     1T  0 stratis /mnt/git
      │   └─stratis-1-210eb8dac6cb440993454dac6a885dae-thin-fs-ef52c734ca1a48739951efd49963924f 253:36   0     1T  0 stratis /mnt/mirror
      └─stratis-1-private-210eb8dac6cb440993454dac6a885dae-flex-mdv                             253:6    0    16M  0 stratis
trgill commented 5 years ago

@ingothierack thanks for the help.

I think the log message was caused by : #1462

The failure to extend the filesystem could have caused by:

  1. xfs_growfs taking a long time. xfs_growfs is typically quick - stratisd depends on it. I was not able to reproduce this problem by causing xfs_growfs to run slowly - but the possibility needs to be addressed and could cause the problem you've described.

  2. creating a new filesystem when a different filesystem needs to be extended. When I was working to reproduce this issue, I saw that filesystem creation is very slow when MD is building the RAID5 checksum #1460 - I think you may have created a new filesystem at the same time another filesystem approached the 1TB limit. I've documented this bug at: https://bugzilla.redhat.com/show_bug.cgi?id=1687974

Do you think either of these situations explain what happened?

Thanks.

ingothierack commented 5 years ago

@trgill hopefully help :)

Yes, would say it explains what happens.

What here comes in my mind, have you a test, where you copy(rsync(here i go sometimes freezes) or cp) 1000000 files with an volume of 2 TB from a slightly faster none stratis filesystem to a stratis filesystem? That's was i had done here.

What i have checked on my side: A RAID5 build did not run. A data-check on this md was running 24 hours before. It took around 9 hours. On thing what here comes to my mind, one of the three disks was faulty(a large range of bad sectors) not visible at this time(disk was now replaced). So this might be also a problems(performance) here.

I think, we can close this here. I than hope for stratis 1.1. As of now, no further problems so far.