search

ceph / ceph-csi

CSI driver for Ceph
Apache License 2.0
1.27k stars 539 forks source link

CephFS CSI provisioner volume failed with exit status 13 #1598

Closed Ayden-Franklin closed 3 years ago

Ayden-Franklin commented 3 years ago

Describe the bug

I created a PVC and it was always Pending.

Environment details

Logs

I tried to execute the command:

# ceph -m 10.0.0.201,10.0.0.202:6789,10.0.0.203:6789 --id admin --keyfile=AQAyTwRftPEIABAA91ru5ajZinTDbNkOqdCUvA== -c /etc/ceph/ceph.conf fs get cephfs --format=json | jq .
{
  "mdsmap": {
    "epoch": 59,
    "flags": 18,
    "ever_allowed_features": 0,
    "explicitly_allowed_features": 0,
    "created": "2020-07-08T02:23:02.469289+0000",
    "modified": "2020-10-17T04:02:23.871619+0000",
    "tableserver": 0,
    "root": 0,
    "session_timeout": 60,
    "session_autoclose": 300,
    "min_compat_client": "0 (unknown)",
    "max_file_size": 1099511627776,
    "last_failure": 0,
    "last_failure_osd_epoch": 2019,
    "compat": {
      "compat": {},
      "ro_compat": {},
      "incompat": {
        "feature_1": "base v0.20",
        "feature_2": "client writeable ranges",
        "feature_3": "default file layouts on dirs",
        "feature_4": "dir inode in separate object",
        "feature_5": "mds uses versioned encoding",
        "feature_6": "dirfrag is stored in omap",
        "feature_8": "no anchor table",
        "feature_9": "file layout v2",
        "feature_10": "snaprealm v2"
      }
    },
    "max_mds": 1,
    "in": [
      0
    ],
    "up": {
      "mds_0": 144105
    },
    "failed": [],
    "damaged": [],
    "stopped": [],
    "info": {
      "gid_144105": {
        "gid": 144105,
        "name": "cephfs.imagego-test-0001.tymuhq",
        "rank": 0,
        "incarnation": 55,
        "state": "up:active",
        "state_seq": 8,
        "addr": "10.0.0.201:6801/722344786",
        "addrs": {
          "addrvec": [
            {
              "type": "v2",
              "addr": "10.0.0.201:6800",
              "nonce": 722344786
            },
            {
              "type": "v1",
              "addr": "10.0.0.201:6801",
              "nonce": 722344786
            }
          ]
        },
        "join_fscid": 1,
        "export_targets": [],
        "features": 4540138292836696000,
        "flags": 0
      }
    },
    "data_pools": [
      4
    ],
    "metadata_pool": 3,
    "enabled": true,
    "fs_name": "cephfs",
    "balancer": "",
    "standby_count_wanted": 1
  },
  "id": 1
}
Ayden-Franklin commented 3 years ago

@Madhu-1 Excuse me, would you like to tell me what is the exact meaning of code = InvalidArgument desc = an error (exit status 13)?

Madhu-1 commented 3 years ago

@Macrame have you executed from the cephfs-plugin container of provisioner pods?

Ayden-Franklin commented 3 years ago 
kubectl exec -it csi-cephfsplugin-provisioner-5c9fff55d6-r5h8c -c csi-cephfsplugin  -- ceph -m 10.0.0.201,10.0.0.202:6789,10.0.0.203:6789 --id admin --keyfile=AQAyTwRftPEIABAA91ru5ajZinTDbNkOqdCUvA== -c /etc/ceph/ceph.conf fs get cephfs --format=json
[errno 1] RADOS permission error (error connecting to the cluster)
command terminated with exit code 1

It looks like the permission issue like #996

Ayden-Franklin commented 3 years ago

Is there any docs to explain how to solve this issue? I read the documentation of Ceph which dosen't mention using CephFS on Kubernetes except RBD

Madhu-1 commented 3 years ago

please check you have required caps https://github.com/ceph/ceph-csi/blob/master/docs/capabilities.md#cephfs and also make sure you pass the correct key.

Ayden-Franklin commented 3 years ago

Thanks. That is what I am looking for. I'll close this if I succeeded to create my PVC.