bch2_write_point_do_index_updates hang [b9bd694]

[leona-ya]

Please search for duplicates

Version

Make sure you're using a reasonably new version.

Provide the commit hash from the kernel version (preferable) or tools, don't say "I'm using the latest master" as that will very quickly become out of date.\=

b9bd69421f7364ca4ff11c827fd0e171a8b826ea

Generic info Provide the output of:

# bcachefs fs usage

Filesystem: 15ccf53c-faaf-44e4-aea8-043a5f6f4d24
Size:                   919199002624
Used:                    99376343040
Online reserved:            94324224

Data type       Required/total  Devices
reserved:       1/0                    [] 42231296
btree:          1/1             [nvme0n1p2]               2393636864
user:           1/1             [nvme0n1p2]              92545379328

(no label) (device 0):     nvme0n1p2              rw
                                data         buckets    fragmented
  free:                            0         1707023
  sb:                        3149824               7        520192
  journal:                4294967296            8192
  btree:                  2393636864            7128    1343488000
  user:                  92545379328          183338    3576534016
  cached:                          0               0
  parity:                          0               0
  stripe:                          0               0
  need_gc_gens:                    0               0
  need_discard:                    0               0
  erasure coded:                   0               0
  capacity:             999129350144         1905688

# bcachefs show-super
External UUID:                              15ccf53c-faaf-44e4-aea8-043a5f6f4d24
Internal UUID:                              8bcded38-66fc-467a-aaad-5999c4834b32
Device index:                               0
Label:
Version:                                    1.3: rebalance_work
Version upgrade complete:                   1.3: rebalance_work
Oldest version on disk:                     1.3: rebalance_work
Created:                                    Wed Nov  8 11:26:28 2023

Sequence number:                            46
Superblock size:                            4512
Clean:                                      0
Devices:                                    1
Sections:                                   members_v1,crypt,replicas_v0,clean,journal_seq_blacklist,journal_v2,counters,members_v2,errors
Features:                                   journal_seq_blacklist_v3,reflink,new_siphash,inline_data,new_extent_overwrite,btree_ptr_v2,extents_above_btree_updates,btree_updates_journalled,reflink_inline_data,new_varint,journal_no_flush,alloc_v2,extents_across_btree_nodes
Compat features:                            alloc_info,alloc_metadata,extents_above_btree_updates_done,bformat_overflow_done

Options:
  block_size:                               512 B
  btree_node_size:                          256 KiB
  errors:                                   continue [ro] panic
  metadata_replicas:                        1
  data_replicas:                            1
  metadata_replicas_required:               1
  data_replicas_required:                   1
  encoded_extent_max:                       64.0 KiB
  metadata_checksum:                        none [crc32c] crc64 xxhash
  data_checksum:                            none [crc32c] crc64 xxhash
  compression:                              none
  background_compression:                   none
  str_hash:                                 crc32c crc64 [siphash]
  metadata_target:                          none
  foreground_target:                        none
  background_target:                        none
  promote_target:                           none
  erasure_code:                             0
  inodes_32bit:                             1
  shard_inode_numbers:                      1
  inodes_use_key_cache:                     1
  gc_reserve_percent:                       8
  gc_reserve_bytes:                         0 B
  root_reserve_percent:                     0
  wide_macs:                                0
  acl:                                      1
  usrquota:                                 0
  grpquota:                                 0
  prjquota:                                 0
  journal_flush_delay:                      1000
  journal_flush_disabled:                   0
  journal_reclaim_delay:                    100
  journal_transaction_names:                1
  version_upgrade:                          [compatible] incompatible none
  nocow:                                    0

members_v2 (size 136):
  Device:                                   0
    Label:                                  (none)
    UUID:                                   07540216-9b1d-4933-acce-c59e9da98d27
    Size:                                   931 GiB
    read errors:                            0
    write errors:                           0
    checksum errors:                        0
    seqread iops:                           0
    seqwrite iops:                          0
    randread iops:                          0
    randwrite iops:                         0
    Bucket size:                            512 KiB
    First bucket:                           0
    Buckets:                                1905688
    Last mount:                             Sun Nov 12 22:07:36 2023

    State:                                  rw
    Data allowed:                           journal,btree,user
    Has data:                               journal,btree,user
    Discard:                                0
    Freespace initialized:                  1

Tools bugs

• pull the latest version, compile it, do not strip the binary.
• provide the exact commands you used to run.
• run with gdb: gdb -ex run --args ./bcacehfs <arguments...>

If you get an assert/segfault etc:

• type bt in to and provide the output here.

If the tools lockup:

• run perf top -p $(pidof bcachefs) and provide a screenshot.
• press ctrl+c to interrupt the process and provide the output of bt.

Kernel bugs Compile the kernel with these flags:

CONFIG_PREEMPT=y
CONFIG_BCACHEFS_DEBUG=y
CONFIG_KALLSYMS=y
CONFIG_KALLSYMS_ALL=y
CONFIG_DEBUG_FS=y
CONFIG_DYNAMIC_FTRACE=y
CONFIG_FTRACE=y

Compiling kernel is not possible with the bug :)

Provide the output of dmesg either in a paste-bin or as attachment, if less than 30~ lines just provide inline here.

https://termbin.com/aqbl

Optional Advanced

If lockup or performance issues:

• run perf record and perf record -e 'bcachefs:*' -o events.data both during the window of issue and then ctrl+c.
• run perf archive to dump symbols.
• archive, compress and upload the files: perf.data, events.data and perf.data.tar.bz2.

Upload large files to a file storage provider:

• provide the output of bcachefs list_journal -a <list of devices> | zstd -f -T0 -o ../journal.log.zst compress & upload all the `metdata.dump.` files from: bcachefs dump -o metadata.dump

[kloenk]

timings: https://termbin.com/d9qe