wonderland
commented
3 months ago Not sure why this is not working for you. I tried the following in a lab, verbose notes below to make it easy to reproduce in your env.
A very simple PVC + Deployment. Container writes timestamps into a file just to simulate regular write access to the PVC:
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: deploy-data
labels:
app: deploy-demo
spec:
accessModes:
- ReadWriteOnce
storageClassName: storage-class-iscsi
resources:
requests:
storage: 10Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: deploy-demo
labels:
app: deploy-demo
spec:
replicas: 1
selector:
matchLabels:
app: deploy-demo
template:
metadata:
labels:
app: deploy-demo
spec:
nodeName: rhel1
containers:
- name: busybox
image: registry.k8s.io/busybox
command: [ "sh", "-c"]
args:
- while true; do
echo -en '\n';
echo "${NODE_NAME}" "`date +"%Y-%m-%d %H:%M:%S"`" >> /data/log.txt;
sleep 1;
done;
env:
- name: NODE_NAME
valueFrom:
fieldRef:
fieldPath: spec.nodeName
volumeMounts:
- mountPath: /data
name: data
volumes:
- name: data
persistentVolumeClaim:
claimName: deploy-dataWhen I apply this, PVC and pod are up and the pod reports the expected PVC size:
k get pod,pvc
NAME READY STATUS RESTARTS AGE
pod/deploy-demo-6bdcb5d994-mz9pg 1/1 Running 0 71s
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE
persistentvolumeclaim/deploy-data Bound pvc-2edea93d-e19d-43b8-a27b-20969bf7ef7b 10Gi RWO storage-class-iscsi <unset> 71s k exec deploy/deploy-demo -- df -h /data
Filesystem Size Used Available Use% Mounted on
/dev/mapper/3600a0980774f6a34712b572d41767175 9.7G 28.0K 9.2G 0% /dataI then modify the PVC yaml to 15GB and apply it again. After a short moment, the PVC reflects the new size. Pod is not restarted:
k get pod,pvc
NAME READY STATUS RESTARTS AGE
pod/deploy-demo-6bdcb5d994-mz9pg 1/1 Running 0 2m12s
NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE
persistentvolumeclaim/deploy-data Bound pvc-2edea93d-e19d-43b8-a27b-20969bf7ef7b 15Gi RWO storage-class-iscsi <unset> 2m12sThe pod sees the new capacity as well (without being restarted):
k exec deploy/deploy-demo -- df -h /data
Filesystem Size Used Available Use% Mounted on
/dev/mapper/3600a0980774f6a34712b572d41767175 14.7G 28.0K 13.9G 0% /dataWhat filesystem type are you using? My setup is on default ext4, not sure if that makes a difference?
k exec deploy/deploy-demo -- mount | grep /data
/dev/mapper/3600a0980774f6a34712b572d41767175 on /data type ext4 (rw,seclabel,relatime,stripe=16)
jwebster7
phhutter
Describe the bug We are encountering an issue where, upon resizing iscsi volumes (ext4), only the volume is being resized, but not the fs itself. We have already tried restarting the application pod that mounts the iscsi volume and attempted to rebind it on a different node using node-selector. However, it seems that resize2fs is not being executed.
I have not been able to find any relevant log files in the trident ds pods. When I perform a manual resize2fs on the volume through the node, the filesystem is successfully resized to the desired size without any errors.
Could you please guide me on how to locate the appropriate logs, or let me know which component is responsible for the fs expansion?
Is my assumption correct, that resizing iscsi volumes necessarily requires a reboot of the application pod?
Environment
To Reproduce Steps to reproduce the behavior:
Expected behavior Upon restart of an application pod which mounts an iscsi volume, a resize2fs is automatically executed.