search

koverstreet / bcachefs

Other
679 stars 70 forks source link

Cannot set metadata_replicas to the number of disks [1e194b3a5924] #565

Open ojab opened 1 year ago

ojab commented 1 year ago

1e194b3a592499e9ec00325091386aa8f995aea3 bcachefs tool version v0.1-700-g1c156d5

4-disk FS, trying to set metadata_replicas 4 via sysfs and it fails:

$ echo 4 | sudo tee /sys/fs/bcachefs/360fc60c-8c44-4f3e-9cc4-fbaeee9e7c3b/options/metadata_replicas
4
tee: /sys/fs/bcachefs/360fc60c-8c44-4f3e-9cc4-fbaeee9e7c3b/options/metadata_replicas: Numerical result out of range
$ echo 3 | sudo tee /sys/fs/bcachefs/360fc60c-8c44-4f3e-9cc4-fbaeee9e7c3b/options/metadata_replicas
3
$

Generic info Provide the output of:

$ bcachefs fs usage -h /mnt/bcachefs/ 
Filesystem: 360fc60c-8c44-4f3e-9cc4-fbaeee9e7c3b
Size:                       34.8 TiB
Used:                       17.4 TiB
Online reserved:                 0 B

Data type       Required/total  Devices
btree:          1/2             [sde1 sdc1]                 55.5 MiB
btree:          1/3             [sda1 sde1 sdc1]            3.96 GiB
btree:          1/2             [sda1 sde1]                  116 GiB
btree:          1/2             [sde1 sdd1]                 49.0 MiB
btree:          1/3             [sda1 sde1 sdd1]            3.92 GiB
user:           1/1             [sda1]                      12.2 TiB
user:           1/2             [sde1 sdd1]                 24.6 MiB
user:           1/2             [sda1 sde1]                  476 GiB
user:           1/1             [sdc1]                      54.5 MiB
user:           1/1             [sde1]                      4.57 TiB
user:           1/1             [sdd1]                      1.74 GiB
cached:         1/1             [sdc1]                      4.97 MiB
cached:         1/1             [sde1]                       274 MiB
cached:         1/1             [sdd1]                      5.00 MiB

hdd.hdd1 (device 0):            sda1              rw
                                data         buckets    fragmented
  free:                          0 B        11627301
  sb:                       3.00 MiB               7       508 KiB
  journal:                  4.00 GiB            8192
  btree:                    60.7 GiB          141045      8.20 GiB
  user:                     12.4 TiB        26105308       134 MiB
  cached:                        0 B               0
  parity:                        0 B               0
  stripe:                        0 B               0
  need_gc_gens:                  0 B               0
  need_discard:                  0 B               0
  erasure coded:                 0 B               0
  capacity:                 18.1 TiB        37881853

hdd.hdd2 (device 1):            sde1              rw
                                data         buckets    fragmented
  free:                          0 B        13817165
  sb:                       3.00 MiB               4      1020 KiB
  journal:                  8.00 GiB            8192
  btree:                    60.7 GiB           75177      12.7 GiB
  user:                     4.81 TiB         5040058      44.7 MiB
  cached:                    274 MiB             330
  parity:                        0 B               0
  stripe:                        0 B               0
  need_gc_gens:                  0 B               0
  need_discard:                  0 B               0
  erasure coded:                 0 B               0
  capacity:                 18.1 TiB        18940926

sdd.ssd1 (device 2):            sdc1              rw
                                data         buckets    fragmented
  free:                          0 B          470315
  sb:                       3.00 MiB               7       508 KiB
  journal:                  1.82 GiB            3726
  btree:                    1.34 GiB            2781      12.8 MiB
  user:                     54.5 MiB             109
  cached:                   4.97 MiB              10
  parity:                        0 B               0
  stripe:                        0 B               0
  need_gc_gens:                  0 B               0
  need_discard:                  0 B               0
  erasure coded:                 0 B               0
  capacity:                  233 GiB          476948

sdd.ssd1 (device 3):            sdd1              rw
                                data         buckets    fragmented
  free:                          0 B          466861
  sb:                       3.00 MiB               7       508 KiB
  journal:                  1.82 GiB            3726
  btree:                    1.33 GiB            2755      14.5 MiB
  user:                     1.75 GiB            3589       228 KiB
  cached:                   5.00 MiB              10
  parity:                        0 B               0
  stripe:                        0 B               0
  need_gc_gens:                  0 B               0
  need_discard:                  0 B               0
  erasure coded:                 0 B               0
  capacity:                  233 GiB          476948
$ bcachefs show-super /dev/sdd1 
External UUID:                              360fc60c-8c44-4f3e-9cc4-fbaeee9e7c3b
Internal UUID:                              bc05affd-9fd1-4eb5-b497-3f7956ac57d2
Device index:                               3
Label:                                      
Version:                                    1.1: snapshot_skiplists
Version upgrade complete:                   1.1: snapshot_skiplists
Oldest version on disk:                     0.29: snapshot_trees
Created:                                    Fri Jun 16 22:38:16 2023
Sequence number:                            212
Superblock size:                            5816
Clean:                                      0
Devices:                                    4
Sections:                                   members,replicas_v0,quota,disk_groups,clean,journal_seq_blacklist,journal_v2,counters
Features:                                   zstd,journal_seq_blacklist_v3,new_siphash,inline_data,new_extent_overwrite,btree_ptr_v2,extents_above_btree_updates,btree_updates_journalled,new_varint,journal_no_flush,alloc_v2,extents_across_btree_nodes
Compat features:                            alloc_info,alloc_metadata,extents_above_btree_updates_done,bformat_overflow_done

Options:
  block_size:                               4.00 KiB
  btree_node_size:                          256 KiB
  errors:                                   continue [ro] panic 
  metadata_replicas:                        2
  data_replicas:                            1
  metadata_replicas_required:               1
  data_replicas_required:                   1
  encoded_extent_max:                       64.0 KiB
  metadata_checksum:                        none crc32c [crc64] xxhash 
  data_checksum:                            none [crc32c] crc64 xxhash 
  compression:                              [none] lz4 gzip zstd 
  background_compression:                   none lz4 gzip [zstd] 
  str_hash:                                 crc32c crc64 [siphash] 
  metadata_target:                          ssd
  foreground_target:                        ssd
  background_target:                        hdd
  promote_target:                           ssd
  erasure_code:                             0
  inodes_32bit:                             1
  shard_inode_numbers:                      1
  inodes_use_key_cache:                     1
  gc_reserve_percent:                       5
  gc_reserve_bytes:                         0 B
  root_reserve_percent:                     0
  wide_macs:                                0
  acl:                                      1
  usrquota:                                 1
  grpquota:                                 1
  prjquota:                                 1
  journal_flush_delay:                      1000
  journal_flush_disabled:                   0
  journal_reclaim_delay:                    100
  journal_transaction_names:                1
  version_upgrade:                          [compatible] incompatible none 
  nocow:                                    0

members (size 232):
  Device:                                   0
    UUID:                                   62f3139c-4515-4e6a-9aa3-24f598263ece
    Size:                                   18.1 TiB
    Bucket size:                            512 KiB
    First bucket:                           0
    Buckets:                                37881853
    Last mount:                             Tue Jul 11 06:09:13 2023
    State:                                  rw
    Label:                                  hdd1 (1)
    Data allowed:                           journal,btree,user
    Has data:                               journal,btree,user
    Discard:                                0
    Freespace initialized:                  1
  Device:                                   1
    UUID:                                   4c1c7eff-f1e9-44b8-bcac-186fb4aa2367
    Size:                                   18.1 TiB
    Bucket size:                            1.00 MiB
    First bucket:                           0
    Buckets:                                18940926
    Last mount:                             Tue Jul 11 06:09:13 2023
    State:                                  rw
    Label:                                  hdd2 (2)
    Data allowed:                           journal,btree,user
    Has data:                               journal,btree,user,cached
    Discard:                                0
    Freespace initialized:                  1
  Device:                                   2
    UUID:                                   145ea1b5-b6c6-4d13-bdee-1d482d29758f
    Size:                                   233 GiB
    Bucket size:                            512 KiB
    First bucket:                           0
    Buckets:                                476948
    Last mount:                             Tue Jul 11 06:09:13 2023
    State:                                  rw
    Label:                                  ssd1 (7)
    Data allowed:                           journal,btree,user
    Has data:                               journal,btree,user,cached
    Discard:                                0
    Freespace initialized:                  1
  Device:                                   3
    UUID:                                   933d8443-8d62-403a-9ef2-e0a3b3fe8c42
    Size:                                   233 GiB
    Bucket size:                            512 KiB
    First bucket:                           0
    Buckets:                                476948
    Last mount:                             Tue Jul 11 06:09:13 2023
    State:                                  rw
    Label:                                  ssd1 (7)
    Data allowed:                           journal,btree,user
    Has data:                               journal,btree,user,cached
    Discard:                                0
    Freespace initialized:                  1
debaba commented 1 year ago

Again both ssds are labeled ssd.ssd1