search

Azure / azure-storage-fuse

A virtual file system adapter for Azure Blob storage
Other
653 stars 206 forks source link

blobfuse doesn't retiurn results for container with large amount of data #265

Closed jpsebasti closed 3 years ago

jpsebasti commented 5 years ago

Which version of the blobfuse was used?

1.0.2

Which OS (please include version) are you using?

Distributor ID: Ubuntu Description: Ubuntu 18.04.2 LTS Release: 18.04 Codename: bionic

What problem was encountered?

No results for ls after a successful mount of a container with a large amount of data. I can mount a container with a smaller amount of data and see the data with ls but when I mount a container with terabytes of data, ls command "thinks" for a long time and returns nothing.

Have you found a mitigation/solution?

no

By default, blobfuse logs errors to syslog. If this is relevant, is there anything in the syslog that might be helpful?

If relevant, please share your mount command.

blobfuse /data1/softlib --tmp-path=/mnt/ramdisk --config-file=/data1/blobfuse_softlib_connection.cfg -o attr_timeout=240 -o entry_timeout=240 -o negative_timeout=120

config file contains: accountName legacymanagementdata accountKey (my account key here) containerName softlib

seguler commented 5 years ago

how many files do you have in the root folder ? We have tested with 3M files and it worked fine.

Could you upgrade to 1.0.3 version and try again ? There is a significant improvement in the version for listing files specifically (both perf and memory optimizations).

jpsebasti commented 5 years ago

Can you tell me how to install 1.0.3? I've tried apt-get install --only-upgrade blobfuse however nothing is updated and version 1.0.2 remains in place. I've also tried apt-get remove blobfuse followed by apt-get install blobfuse however only 1.0.2 is installed. How do I get 1.0.3 version?

seguler commented 5 years ago

Sorry - at this moment you have to clone the repository from master and build from source code. We are about to release 1.0.3 binary to the apt repository but we are blocked due to an issue in our release pipeline.

seguler commented 5 years ago

Package 1.0.3 has been released on Ubuntu. Go ahead and install blobfuse using the following now:

sudo apt-get update
sudo apt-get install blobfuse
jpsebasti commented 5 years ago

Hi, thanks. I'm trying to install blobfuse 1.0.3 and getting this message: $ sudo apt-get install blobfuse Reading package lists... Done Building dependency tree Reading state information... Done The following NEW packages will be installed: blobfuse 0 upgraded, 1 newly installed, 0 to remove and 70 not upgraded. 1 not fully installed or removed. Need to get 0 B/5068 kB of archives. After this operation, 17.2 MB of additional disk space will be used. Selecting previously unselected package blobfuse. (Reading database ... 223653 files and directories currently installed.) Preparing to unpack .../blobfuse_1.0.3_amd64.deb ... Unpacking blobfuse (1.0.3) ... Setting up aadlogin (1.0.008450001) ... Creating the aad_admins group if it is not already there Adding the NSS provider sed: can't read /etc/nsswitch.conf: No such file or directory dpkg: error processing package aadlogin (--configure): installed aadlogin package post-installation script subprocess returned error e xit status 2 Setting up blobfuse (1.0.3) ... Errors were encountered while processing: aadlogin E: Sub-process /usr/bin/dpkg returned an error code (1)

jpsebasti commented 5 years ago

hey nevermind, I have it working

jpsebasti commented 5 years ago

Unfortunately, I get the same results with 1.0.3 of blobfuse.

I mount the the blob: export AZURE_STORAGE_ACCOUNT=storageaccountname export AZURE_STORAGE_ACCESS_KEY=key

blobfuse /data1/softlib --tmp-path=/mnt/blobfusetmp -o attr_timeout=240 -o entry_timeout=240 -o negative_timeout=120 --container-name=softlib --log-level=LOG_DEBUG --file-cache-timeout-in-seconds=120

Thinks about it for a while and then returns my prompt so I'm guessing it was a success.

Now I try to see what's in the blob container:

$ cd /data1/softlib/ $ ls -ltr total 0

Some information on the container I am trying to mount: There are 504,920 blobs These blobs contain: 951,013,076,490 bytes

NaraVen commented 4 years ago

What is the memory in your VM?

NaraVen commented 3 years ago

We have tested with 240 petabytes of data on GPU VMs and they do mount and we were able to read files. This is in the latest version 1.3.4. Could you please upgrade and try. If you still run into mounting problems please file a new issue.