Azure / azure-storage-fuse

A virtual file system adapter for Azure Blob storage
Other
674 stars 209 forks source link

The blob file will not updated if reading the same file from the different workload #1525

Closed zhangchl007 closed 2 months ago

zhangchl007 commented 2 months ago

Which version of blobfuse was used?

blobfuse2-2.3.0-1

Which OS distribution and version are you using?

azurelinux

If relevant, please share your mount command.

the pvc mount by storage class

---
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: azureblob-fuse2
provisioner: blob.csi.azure.com
parameters:
  skuName: Premium_LRS  # available values: Standard_LRS, Premium_LRS, Standard_GRS, Standard_RAGRS, Standard_ZRS, Premium_ZRS
  protocol: fuse2
reclaimPolicy: Delete
volumeBindingMode: Immediate
allowVolumeExpansion: true
mountOptions:
  - -o allow_other
  - --file-cache-timeout-in-seconds=1
  - --use-attr-cache=true
  - --cancel-list-on-mount-seconds=1
  - -o attr_timeout=1
  - -o entry_timeout=1
  - -o negative_timeout=120
  - --log-level=LOG_WARNING
  - --cache-size-mb=10000

What was the issue encountered?

we mount the blobfuse into different pods with the same pvc in different AKS nodes , one pod will update the data in the file , another pod only read the file, but we find the

Have you found a mitigation/solution?

not yet

Please share logs if available.

image

image

souravgupta-msft commented 2 months ago

Hi @zhangchl007. This is a multiple reader/writer scenario. You will have to disable caching in this case. Please add direct_io in your config to disable kernel cache and also disable attribute and file cache.

mountOptions:
  - -o allow_other
  - -o direct_io
  - --file-cache-timeout-in-seconds=0
  - --use-attr-cache=false
  - --cancel-list-on-mount-seconds=1
  - -o attr_timeout=0
  - -o entry_timeout=0
  - -o negative_timeout=0
  - --log-level=LOG_WARNING
  - --cache-size-mb=0
zhangchl007 commented 2 months ago

@souravgupta-msft thanks, let me try it

souravgupta-msft commented 2 months ago

Closing this, as it is working as expected after disabling caching (kernel as well as file and attribute cache).