search

seaweedfs / seaweedfs

SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.
https://seaweedfs.com
Apache License 2.0
22.46k stars 2.26k forks source link

[vacuum] Error when committing vacuum 706: rename /opt/seaweedfs_volume/disk21/bucket_706.cpd no such file or directory #5660

Open kmlebedev opened 3 months ago

kmlebedev commented 3 months ago

Describe the bug

When using the directory to store .idx files option -dir.idx

/usr/bin/weed -v volume -dir /opt/seaweedfs_volume/disk21 -dir.idx /opt/seaweedfs_volume_idx/volume21
 ls -lh /opt/seaweedfs_volume_idx/volume21/*706*
-rw-r--r-- 1 seaweed seaweed 70K Jun  7 08:09 /opt/seaweedfs_volume_idx/volume21/bucket_706.cpx
-rw-r--r-- 1 seaweed seaweed 71K Jun  7 08:15 /opt/seaweedfs_volume_idx/volume21/bucket_706.idx

error log:

I0607 07:16:50.859671 needle_map_memory.go:111 loading idx from offset 0 for file: /opt/seaweedfs_volume_idx/volume21/bucket_706.cpx
I0607 07:27:24.683905 volume_vacuum.go:106 Committing volume 706 vacuuming...
I0607 07:27:24.705383 volume_loading.go:91 readSuperBlock volume 706 version 3
I0607 07:27:24.721997 volume_loading.go:139 updating memory compact index /opt/seaweedfs_volume_idx/volume21/bucket_706.idx
I0607 08:09:49.268803 needle_map_memory.go:111 loading idx from offset 0 for file: /opt/seaweedfs_volume_idx/volume21/bucket_706.cpx
I0607 08:19:10.490827 volume_vacuum.go:106 Committing volume 706 vacuuming...
E0607 08:19:10.497500 volume_grpc_vacuum.go:89 failed commit volume 706: rename /opt/seaweedfs_volume/disk21/bucket_706.cpd: rename /opt/seaweedfs_volume/disk21/bucket_706.cpd /opt/seaweedfs_volume/disk21/bucket_706.dat: no such file or directory

System Setup

weed version
version 8000GB 3.65 0a645057 linux amd64
kmlebedev commented 3 months ago

Since the beginning of the file is filled with zeros, the compact cannot be completed, but at the same time the commit was processed, but the cpd file was not created https://github.com/seaweedfs/seaweedfs/issues/5674