After the cache disk is recovered, write data times out for 5 minutes

[YunhuiChen]

What happened:

Use writeback mode

1、fio randwrite fio --name=big-file-multi-write --directory=/var/lib/juicefs/volume/pvc-618dce8f-e2a1-4847-93e8-8a4a7ce247b9-ttbxpf --rw=randwrite --refill_buffers --bs=4k --size=1G --numjobs=1 --end_fsync=1

2、Simulated cache disk failure

• name: juicefs-io-fault templateType: IOChaos ioChaos: action: fault mode: one selector: namespaces:
  • kube-system labelSelectors: app.kubernetes.io/name: juicefs-mount labelSelectors: chaostest: "true" volumePath: /data/cache path: /data/cache/*/ errno: 5 percent: 1

3、fio is stuck. Return to EIO

4、Restore cache disk, wait a few minutes to write fio again, still stuck to return to EIO log.tar.gz [Uploading log.tar.gz…]()

5、But after waiting for a long time, I can read and write normally

What you expected to happen:

How to reproduce it (as minimally and precisely as possible):

Anything else we need to know?

Environment:

• JuiceFS version (use juicefs --version) or Hadoop Java SDK version:
• Cloud provider or hardware configuration running JuiceFS:
• OS (e.g cat /etc/os-release):
• Kernel (e.g. uname -a):
• Object storage (cloud provider and region, or self maintained):
• Metadata engine info (version, cloud provider managed or self maintained):
• Network connectivity (JuiceFS to metadata engine, JuiceFS to object storage):
• Others:

[davies]

Should be fixed by #4871