Panic when formatting s3 buckets in us-east1

mwqgithub commented 5 months ago

What happened:

[root@ip-172-31-45-198 ~]# juicefs format sqlite3://myjfs.db myjfs --storage s3 --bucket juicefs-us-east-1 2024/04/07 03:11:20.228937 juicefs[13600] : Meta address: sqlite3://myjfs.db [interface.go:497] panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x1ea69e9]

goroutine 1 [running]: github.com/juicedata/juicefs/pkg/object.autoS3Region({0xc000112788, 0x11}, {0x0?, 0x1?}, {0x0?, 0xc000db2fa8?}) /go/src/github.com/juicedata/juicefs/pkg/object/s3.go:394 +0x8a9 github.com/juicedata/juicefs/pkg/object.newS3({0xc000844018, 0x11}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}) /go/src/github.com/juicedata/juicefs/pkg/object/s3.go:468 +0x32b github.com/juicedata/juicefs/pkg/object.CreateStorage({0xc00005212e, 0x2}, {0xc000844018, 0x11}, {0x0, 0x0}, {0x0, 0x0}, {0x0, 0x0}) /go/src/github.com/juicedata/juicefs/pkg/object/object_storage.go:172 +0x175 github.com/juicedata/juicefs/cmd.createStorage({{0xc000052120, 0x5}, {0xc00084c360, 0x24}, {0xc00005212e, 0x2}, {0x0, 0x0}, {0xc000844018, 0x11}, ...}) /go/src/github.com/juicedata/juicefs/cmd/format.go:239 +0x50a github.com/juicedata/juicefs/cmd.format(0xc0002b7180) /go/src/github.com/juicedata/juicefs/cmd/format.go:467 +0xf25 github.com/urfave/cli/v2.(Command).Run(0xc0010ee120, 0xc001135c80) /go/pkg/mod/github.com/juicedata/cli/v2@v2.19.4-0.20230605075551-9c9c5c0dce83/command.go:177 +0x719 github.com/urfave/cli/v2.(App).RunContext(0xc000aeba40, {0x3874230?, 0xc000052088}, {0xc000eca200, 0x8, 0x8}) /go/pkg/mod/github.com/juicedata/cli/v2@v2.19.4-0.20230605075551-9c9c5c0dce83/app.go:389 +0x1035 github.com/urfave/cli/v2.(*App).Run(...) /go/pkg/mod/github.com/juicedata/cli/v2@v2.19.4-0.20230605075551-9c9c5c0dce83/app.go:254 github.com/juicedata/juicefs/cmd.Main({0xc000eca000?, 0x8?, 0x8?}) /go/src/github.com/juicedata/juicefs/cmd/main.go:94 +0x354c main.main() /go/src/github.com/juicedata/juicefs/main.go:29 +0x2e [root@ip-172-31-45-198 ~]#

What you expected to happen:

Format a volume with the bucket juicefs-us-east-1.

How to reproduce it (as minimally and precisely as possible):

Format a volume with a s3 bucket in us-east-1

Anything else we need to know?

GetBucketLocation will return nil in us-east-1.

Environment:

JuiceFS version (use juicefs --version) or Hadoop Java SDK version:

[root@ip-172-31-45-198 ~]# juicefs --version juicefs version 1.1.2+2024-02-04.8dbd89a

Cloud provider or hardware configuration running JuiceFS:

Cloud provider: aws Instance Type: c5.xlarge

OS (e.g cat /etc/os-release):

[root@ip-172-31-45-198 ~]# cat /etc/os-release NAME="Rocky Linux" VERSION="9.3 (Blue Onyx)" ID="rocky" ID_LIKE="rhel centos fedora" VERSION_ID="9.3" PLATFORM_ID="platform:el9" PRETTY_NAME="Rocky Linux 9.3 (Blue Onyx)" ANSI_COLOR="0;32" LOGO="fedora-logo-icon" CPE_NAME="cpe:/o:rocky:rocky:9::baseos" HOME_URL="https://rockylinux.org/" BUG_REPORT_URL="https://bugs.rockylinux.org/" SUPPORT_END="2032-05-31" ROCKY_SUPPORT_PRODUCT="Rocky-Linux-9" ROCKY_SUPPORT_PRODUCT_VERSION="9.3" REDHAT_SUPPORT_PRODUCT="Rocky Linux" REDHAT_SUPPORT_PRODUCT_VERSION="9.3"

Kernel (e.g. uname -a):

[root@ip-172-31-45-198 ~]# uname -a Linux ip-172-31-45-198.ec2.internal 5.14.0-362.8.1.el9_3.x86_64 #1 SMP PREEMPT_DYNAMIC Wed Nov 8 17:36:32 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

Object storage (cloud provider and region, or self maintained):

s3

Metadata engine info (version, cloud provider managed or self maintained):

sqlite

Network connectivity (JuiceFS to metadata engine, JuiceFS to object storage):
Others:

zhijian-pro commented 5 months ago

juicefs format \
    --storage minio \
    --bucket http://127.0.0.1:9000/<bucket> \
    --access-key minioadmin \
    --secret-key minioadmin \
    ... \
    myjfs

Your bucket URL format may not be correct, please refer to the document below to set the bucket URL correctly. https://juicefs.com/docs/community/reference/how_to_set_up_object_storage/#minio

mwqgithub commented 5 months ago

juicefs format \
    --storage minio \
    --bucket http://127.0.0.1:9000/<bucket> \
    --access-key minioadmin \
    --secret-key minioadmin \
    ... \
    myjfs
Your bucket URL format may not be correct, please refer to the document below to set the bucket URL correctly. https://juicefs.com/docs/community/reference/how_to_set_up_object_storage/#minio

Change bucket URL worked. Thanks for your help!

juicedata / juicefs

Panic when formatting s3 buckets in us-east1 #4649