Multiple, started out using your docker containers, then also built my own container from your repo, and now have just installed it on Arch Linux on my desktop for debugging purposes from
https://aur.archlinux.org/packages/azure-storage-fuse
If relevant, please share your mount command.
blobfuse2 mount /mnt/azure --config-file=/home/matt/.blobfuse2/config.yaml
What was the issue encountered?
I cannot see folders at all. I was wanting to mount a blob container that has like 2k folders, but I see exactly 0.
I can go into the blobfuse cache, make a folder, and it appears in azure, but none of the folders that only exist in azure ever show up.
As far as I've been able to understand it, running "ls" in my MOUNT folder, will then cause blobfuse2 to load the files from azure blob, which show up in the CACHE folder and I think then get symlinked into the MOUNT folder, and after a few moments of not interacting with the file, it all gets cleaned up. I can see this pattern when I mount containers with just "blobs/files" and ls them, but when interacting with a container that has folders at the top level, I see nothing at all ever.
Here's me making a test folder inside of the cache, mostly just trying to test blobfuse2 and understand it.
Have you found a mitigation/solution?
No. I have mounted other containers that simply have blobs/files inside of them, and I do see those. It's simply folders that I cannot see. Like in the above screenshot, no matter what I do I cannot get the folders 1101, 1102, or 6201 to show on my host, be it in the cache or the mount folder.
Please share logs if available.
Please ignore the "invalid prefetch count" and folder permissions errors at the beginning of the logs, I was still trying to figure out a config.yaml 100% at that point.
Wed Oct 2 13:28:22 MDT 2024 : blobfuse2[4131458] : [/mnt/azure] LOG_CRIT [mount.go (432)]: Starting Blobfuse2 Mount : 2.3.2 on [Arch Linux]
Wed Oct 2 13:28:22 MDT 2024 : blobfuse2[4131458] : [/mnt/azure] LOG_CRIT [mount.go (434)]: Logging level set to : LOG_WARNING
Wed Oct 2 13:28:22 MDT 2024 : blobfuse2[4131458] : [/mnt/azure] LOG_ERR [block_cache.go (230)]: BlockCache::Configure : Prefetch count can not be less then 11
Wed Oct 2 13:28:22 MDT 2024 : blobfuse2[4131458] : [/mnt/azure] LOG_ERR [pipeline.go (69)]: Pipeline: error creating pipeline component block_cache [config error in block_cache [invalid prefetch count]]
Wed Oct 2 13:28:22 MDT 2024 : blobfuse2[4131458] : [/mnt/azure] LOG_ERR [mount.go (442)]: mount : failed to initialize new pipeline [config error in block_cache [invalid prefetch count]]
Wed Oct 2 13:28:44 MDT 2024 : blobfuse2[4132089] : [/mnt/azure] LOG_CRIT [mount.go (432)]: Starting Blobfuse2 Mount : 2.3.2 on [Arch Linux]
Wed Oct 2 13:28:44 MDT 2024 : blobfuse2[4132089] : [/mnt/azure] LOG_CRIT [mount.go (434)]: Logging level set to : LOG_WARNING
Wed Oct 2 13:28:44 MDT 2024 : blobfuse2[4132089] : [/mnt/azure] LOG_ERR [block_cache.go (230)]: BlockCache::Configure : Prefetch count can not be less then 11
Wed Oct 2 13:28:44 MDT 2024 : blobfuse2[4132089] : [/mnt/azure] LOG_ERR [pipeline.go (69)]: Pipeline: error creating pipeline component block_cache [config error in block_cache [invalid prefetch count]]
Wed Oct 2 13:28:44 MDT 2024 : blobfuse2[4132089] : [/mnt/azure] LOG_ERR [mount.go (442)]: mount : failed to initialize new pipeline [config error in block_cache [invalid prefetch count]]
Wed Oct 2 13:28:56 MDT 2024 : blobfuse2[4132453] : [/mnt/azure] LOG_CRIT [mount.go (432)]: Starting Blobfuse2 Mount : 2.3.2 on [Arch Linux]
Wed Oct 2 13:28:56 MDT 2024 : blobfuse2[4132453] : [/mnt/azure] LOG_CRIT [mount.go (434)]: Logging level set to : LOG_WARNING
Wed Oct 2 13:28:56 MDT 2024 : blobfuse2[4132453] : [/mnt/azure] LOG_ERR [block_cache.go (252)]: BlockCache: config error creating directory after clean [mkdir /mnt/blobfuse_cache: permission denied]
Wed Oct 2 13:28:56 MDT 2024 : blobfuse2[4132453] : [/mnt/azure] LOG_ERR [pipeline.go (69)]: Pipeline: error creating pipeline component block_cache [config error in block_cache [mkdir /mnt/blobfuse_cache: permission denied]]
Wed Oct 2 13:28:56 MDT 2024 : blobfuse2[4132453] : [/mnt/azure] LOG_ERR [mount.go (442)]: mount : failed to initialize new pipeline [config error in block_cache [mkdir /mnt/blobfuse_cache: permission denied]]
Wed Oct 2 13:30:02 MDT 2024 : blobfuse2[4134284] : [/mnt/azure] LOG_CRIT [mount.go (432)]: Starting Blobfuse2 Mount : 2.3.2 on [Arch Linux]
Wed Oct 2 13:30:02 MDT 2024 : blobfuse2[4134284] : [/mnt/azure] LOG_CRIT [mount.go (434)]: Logging level set to : LOG_WARNING
Wed Oct 2 13:30:02 MDT 2024 : blobfuse2[4134342] : [/mnt/azure] LOG_CRIT [mount.go (432)]: Starting Blobfuse2 Mount : 2.3.2 on [Arch Linux]
Wed Oct 2 13:30:02 MDT 2024 : blobfuse2[4134342] : [/mnt/azure] LOG_CRIT [mount.go (434)]: Logging level set to : LOG_WARNING
Wed Oct 2 13:30:10 MDT 2024 : blobfuse2[4134342] : [/mnt/azure] LOG_WARNING [block_blob.go (499)]: BlockBlob::getAttrUsingList : blob .nvmrc does not exist
Wed Oct 2 13:30:10 MDT 2024 : blobfuse2[4134342] : [/mnt/azure] LOG_WARNING [block_blob.go (499)]: BlockBlob::getAttrUsingList : blob .conda_config does not exist
Wed Oct 2 13:30:10 MDT 2024 : blobfuse2[4134342] : [/mnt/azure] LOG_WARNING [block_blob.go (499)]: BlockBlob::getAttrUsingList : blob .git does not exist
Wed Oct 2 13:30:10 MDT 2024 : blobfuse2[4134342] : [/mnt/azure] LOG_WARNING [block_blob.go (499)]: BlockBlob::getAttrUsingList : blob HEAD does not exist
Wed Oct 2 13:32:04 MDT 2024 : blobfuse2[4134342] : [/mnt/azure] LOG_WARNING [block_blob.go (499)]: BlockBlob::getAttrUsingList : blob test does not exist
Wed Oct 2 13:32:04 MDT 2024 : blobfuse2[4134342] : [/mnt/azure] LOG_WARNING [block_blob.go (499)]: BlockBlob::getAttrUsingList : blob test does not exist
MatthewABrantley
commented
1 month ago
Which version of blobfuse was used?
2.3.2
Which OS distribution and version are you using?
Multiple, started out using your docker containers, then also built my own container from your repo, and now have just installed it on Arch Linux on my desktop for debugging purposes from https://aur.archlinux.org/packages/azure-storage-fuse
If relevant, please share your mount command.
blobfuse2 mount /mnt/azure --config-file=/home/matt/.blobfuse2/config.yaml
What was the issue encountered?
I cannot see folders at all. I was wanting to mount a blob container that has like 2k folders, but I see exactly 0.
I can go into the blobfuse cache, make a folder, and it appears in azure, but none of the folders that only exist in azure ever show up.
As far as I've been able to understand it, running "ls" in my MOUNT folder, will then cause blobfuse2 to load the files from azure blob, which show up in the CACHE folder and I think then get symlinked into the MOUNT folder, and after a few moments of not interacting with the file, it all gets cleaned up. I can see this pattern when I mount containers with just "blobs/files" and ls them, but when interacting with a container that has folders at the top level, I see nothing at all ever.
Here's me making a test folder inside of the cache, mostly just trying to test blobfuse2 and understand it.
Have you found a mitigation/solution?
No. I have mounted other containers that simply have blobs/files inside of them, and I do see those. It's simply folders that I cannot see. Like in the above screenshot, no matter what I do I cannot get the folders 1101, 1102, or 6201 to show on my host, be it in the cache or the mount folder.
Please share logs if available.
Please ignore the "invalid prefetch count" and folder permissions errors at the beginning of the logs, I was still trying to figure out a config.yaml 100% at that point.