Ceph RBD csi

The RBD CSI plugin is able to provision new RBD images and attach and mount those to workloads.

Configuration

Available command line arguments:

Option Default value Description
--endpoint unix:///tmp/csi.sock CSI endpoint, must be a UNIX socket
--csi-addons-endpoint unix:///tmp/csi-addons.sock CSI-Addons endpoint, must be a UNIX socket
--drivername rbd.csi.ceph.com Name of the driver (Kubernetes: provisioner field in StorageClass must correspond to this value)
--nodeid empty This node’s ID
--type empty Driver type: [rbd/cephfs]. If the driver type is set to rbd it will act as a rbd plugin or if it’s set to cephfs will act as a cephfs plugin
--instanceid “default” Unique ID distinguishing this instance of Ceph CSI among other instances, when sharing Ceph clusters across CSI instances for provisioning
--pidlimit 0 Configure the PID limit in cgroups. The container runtime can restrict the number of processes/tasks which can cause problems while provisioning (or deleting) a large number of volumes. A value of -1 configures the limit to the maximum, 0 does not configure limits at all.
--metricsport 8080 TCP port for liveness metrics requests
--metricspath "/metrics" Path of prometheus endpoint where metrics will be available
--polltime "60s" Time interval in between each poll
--timeout "3s" Probe timeout in seconds
--clustername empty Cluster name to set on RBD image
--domainlabels empty Kubernetes node labels to use as CSI domain labels for topology aware provisioning, should be a comma separated value (ex:= “failure-domain/region,failure-domain/zone”)
--rbdhardmaxclonedepth 8 Hard limit for maximum number of nested volume clones that are taken before a flatten occurs
--rbdsoftmaxclonedepth 4 Soft limit for maximum number of nested volume clones that are taken before a flatten occurs
--skipforceflatten false skip image flattening on kernel < 5.2 which support mapping of rbd images which has the deep-flatten feature
--maxsnapshotsonimage 450 Maximum number of snapshots allowed on rbd image without flattening
--setmetadata true Deprecated. Set metadata on volume. This flag will be removed in a future release.
--enable-read-affinity false enable read affinity
--crush-location-labels empty Kubernetes node labels that determine the CRUSH location the node belongs to, separated by ‘,’. Note: These labels will be replaced if crush location labels are defined in the ceph-csi-config ConfigMap for the specific cluster.
--logslowopinterval 30s Log slow operations at the specified rate. Operation is considered slow if it outlives its deadline.
--feature-gates empty Comma-separated list of feature gates (e.g., SlowGRPCRestart=false). Available gates: SlowGRPCRestart (default: true) — restart the process when a unary gRPC call is stuck for more than 10 minutes.

Available volume parameters:

Parameter Required Description
clusterID yes String representing a Ceph cluster, must be unique across all Ceph clusters in use for provisioning, cannot be greater than 36 bytes in length, and should remain immutable for the lifetime of the Ceph cluster in use
pool yes Ceph pool into which the RBD image shall be created
dataPool no Ceph pool used for the data of the RBD images.
volumeNamePrefix no Prefix to use for naming RBD images (defaults to csi-vol-).
snapshotNamePrefix no Prefix to use for naming RBD snapshot images (defaults to csi-snap-).
imageFeatures no RBD image features. CSI RBD currently supports layering, journaling, exclusive-lock, object-map, fast-diff, deep-flatten features. deep-flatten is added for cloned images. Refer https://docs.ceph.com/en/latest/rbd/rbd-config-ref/#image-features for image feature dependencies.
mkfsOptions no Options to pass to the mkfs command while creating the filesystem on the RBD device. Check the man-page for the mkfs command for the filesystem for more details. When mkfsOptions is set here, the defaults will not be used, consider including them in this parameter.
tryOtherMounters no Specifies whether to try other mounters in case if the current mounter fails to mount the rbd image for any reason
mapOptions no Map options to use when mapping rbd image. See krbd and nbd options.
unmapOptions no Unmap options to use when unmapping rbd image. See krbd and nbd options.
csi.storage.k8s.io/provisioner-secret-name, csi.storage.k8s.io/node-stage-secret-name yes (for Kubernetes) name of the Kubernetes Secret object containing Ceph client credentials. Both parameters should have the same value
csi.storage.k8s.io/provisioner-secret-namespace, csi.storage.k8s.io/node-stage-secret-namespace yes (for Kubernetes) namespaces of the above Secret objects
mounter no if set to rbd-nbd, use rbd-nbd on nodes that have rbd-nbd and nbd kernel modules to map rbd images
encrypted no disabled by default, use "true" to enable either LUKS or fscrypt encryption on PVC and "false" to disable it. Do not change for existing storageclasses
encryptionSectorSize no set the sector size that is used to perform I/O. Size must be a power of two and in between 512 and 4096. Typical values are 4096 and 512.
encryptionCipher no set the cipher that is used for the volume encryption. Supported Keywords: aes-xts-plain64 (default), serpent-xts-plain64, aes-xts-random, serpent-xts-random
encryptionKeySize no set the key size used for encryption. Typical key sizes are 128, 256 and 512. Default 256 (or 512 when using an xts cipher mode)
integrityMode no set and enable the integrity verification for volume encryption. Supported Keywords: hmac-sha256, hmac-sha512
encryptionKMSID no required if encryption is enabled and a kms is used to store passphrases
encryptionType no Either block or file. If unset or block use LUKS block device encryption. If file use ext4 fscrypt to encrypt on the file system level (requires kernel support).
stripeUnit no stripe unit in bytes
stripeCount no objects to stripe over before looping
objectSize no object size in bytes
baseIops no the base limit of operations per second
maxIops no the max limit of operations per second
baseReadIops no the base limit of read operations per second
maxReadIops no the max limit of read operations per second
baseWriteIops no the base limit of write operations per second
maxWriteIops no the max limit of write operations per second
baseBps no the base limit of bytes per second
maxBps no the max limit of bytes per second
baseReadBps no the base limit of read bytes per second
maxReadBps no the max limit of read bytes per second
baseWriteBps no the base limit of write bytes per second
maxWriteBps no the max limit of write bytes per second
iopsPerGiB no the limit of operations per GiB
readIopsPerGiB no the limit of read operations per GiB
writeIopsPerGiB no the limit of write operations per GiB
bpsPerGiB no the limit of bytes per GiB
readBpsPerGiB no the limit of read bytes per GiB
writeBpsPerGiB no the limit of write bytes per GiB
baseVolSizeBytes no the min size of volume what use to calculate qos beased on capacity
extraDeploy no array of extra objects to deploy with the release

NOTE: An accompanying CSI configuration file, needs to be provided to the running pods. Refer to Creating CSI configuration for more information.

NOTE: A suggested way to populate and retain uniqueness of the clusterID is to use the output of ceph fsid of the Ceph cluster to be used for provisioning.

Required secrets:

User credentials, with required access to the pool being used in the storage class, is required for provisioning new RBD images.

Deployment

Use the rbd templates(/path/to/ceph-csi/deploy/rbd/kubernetes)

Your Kubernetes cluster must allow privileged pods (i.e. --allow-privileged flag must be set to true for both the API server and the kubelet). Moreover, as stated in the mount propagation docs, the Docker daemon of the cluster nodes must allow shared mounts.

YAML manifests are located in deploy/rbd/kubernetes.

Create CSIDriver object:

1
kubectl create -f csidriver.yaml

Deploy RBACs for sidecar containers and node plugins:

1
2
kubectl create -f csi-provisioner-rbac.yaml
kubectl create -f csi-nodeplugin-rbac.yaml

Those manifests deploy service accounts, cluster roles and cluster role bindings. These are shared for both RBD and CephFS CSI plugins, as they require the same permissions.

Deploy ConfigMap for CSI plugins:

1
kubectl create -f csi-config-map.yaml

The configmap deploys an empty CSI configuration that is mounted as a volume within the Ceph CSI plugin pods. To add a specific Ceph clusters configuration details, refer to Creating CSI configuration for RBD based provisioning for more information.

Deploy Ceph configuration ConfigMap for CSI pods:

1
kubectl create -f ../../ceph-conf.yaml

Deploy prerequisites for CSI Snapshot:

If you intend to use the snapshot functionality in Kubernetes cluster, please refer to snap-clone.md

Deploy CSI sidecar containers:

1
kubectl create -f csi-rbdplugin-provisioner.yaml

Deploys deployment of provision which includes external-provisioner ,external-attacher,csi-snapshotter sidecar containers and CSI RBD plugin.

Deploy RBD CSI driver:

1
kubectl create -f csi-rbdplugin.yaml

Deploys a daemon set with two containers: CSI node-driver-registrar and the CSI RBD driver.

NOTE: In case you want to use a different release version, replace canary with the release version in the provisioner and nodeplugin YAMLs.

1
2
# for stable functionality replace canary with latest release version
image: quay.io/cephcsi/cephcsi:canary

Check the release version here.

Verify

After successfully completing the steps above, you should see output similar to this:

1
2
3
4
5
6
7
8
$ kubectl get all
NAME READY STATUS RESTARTS AGE
pod/csi-rbdplugin-fptqr 3/3 Running 0 21s
pod/csi-rbdplugin-provisioner-0 5/5 Running 0 22s

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/csi-rbdplugin-provisioner ClusterIP 10.104.2.130 <none> 8080/TCP 23s
...

Once the CSI plugin configuration is updated with details from a Ceph cluster of choice, you can try deploying a demo pod from examples/rbd using the instructions provided to test the deployment further.

They are two halves of the same RBD CSI driver: node I/O vs cluster control plane. Both must run; a PVC needs the provisioner to create the image and the plugin to map it on the node.[1][2]

Split of work

csi-rbdplugin.yaml csi-rbdplugin-provisioner.yaml
Kind DaemonSet Deployment
Where Every worker (ubuntu2, ubuntu3) 2 replicas, not per node
CSI role Node service Controller service
Does rbd map / unmap, mkfs, mount into the pod rbd create / delete, expand, snapshot, attach metadata
Host paths /dev, /sys, /lib/modules, kubelet plugin dir Mostly ConfigMaps + emptyDir; /dev optional
Scale One pod per node 2 is enough (leader election)

CreateVolume is the provisioner. NodeStageVolume / NodePublishVolume is the node plugin. That is why timezone /dev mounts belong on the plugin DaemonSet, not as a third provisioner replica.

csi-rbdplugin.yaml (node):

  • csi-rbdplugin — talks to kubelet via the CSI socket, maps the image
  • driver-registrar — registers rbd.csi.ceph.com with kubelet
  • liveness-prometheus — health

Without this on a node, the PVC binds but the pod stays ContainerCreating (FailedMount).

csi-rbdplugin-provisioner.yaml (controller, your 7/7 pod):

  • csi-provisioner — watches PVCs, calls CreateVolume / DeleteVolume
  • csi-attacher — VolumeAttachment
  • csi-resizer — PVC expand
  • csi-snapshotter — VolumeSnapshot
  • csi-rbdplugin — controller-side Ceph client (same binary, different flags)
  • extra sidecars (liveness / CSI-Addons)

Without this, PVCs stay Pending (waiting for a volume to be created).

storage class

1
2
3
4
kubectl create -f secret.yaml
kubectl create -f storageclass.yaml
kubectl create -f pvc.yaml
kubectl create -f pod.yaml

PVC flows

1
2
3
PVC → provisioner CreateVolume → RBD image in kubernetes-meta (data on kubernetes-ec)
→ attacher
→ node plugin on the pod’s node: rbd map + mount