Azure / azure-storage-fuse

A virtual file system adapter for Azure Blob storage
Other
647 stars 204 forks source link

Azure blob fuse via a proxy #1402

Closed thekoolestkat closed 1 month ago

thekoolestkat commented 2 months ago

Which version of blobfuse was used? Version 2.2.1

Which OS distribution and version are you using? RedHat 8.9

If relevant, please share your mount command.

What was the issue encountered? I'm having issues with Proxy and azure storage fuse 2.2.1 trying to mount a blob store via proxy. I can see the connection going out to do a version check but it never attempts to connect to azure storage blob. When I disable version check it doesn't even try to go out. If I disable proxy settings the connections works perfectly.

Have you found a mitigation/solution?No

Please share logs if available.

souravgupta-msft commented 2 months ago

Hi @thekoolestkat. Thanks for reaching out. Can you share your config file and mount command? You can mention the proxy in your config file in either http-proxy or https-proxy under azstorage section.

thekoolestkat commented 2 months ago

Hi Thanks for the prompt reply the mount command i used was "blobfuse2 mount ./blobtestrhel --config-file=./fuse-config.yaml" i also tried "blobfuse2 mount ./blobtestrhel --config-file=./fuse-config.yaml --disable-version-check=true" the proxy settings are on the linux host not held in the config file.

Config File:

logging:

type: syslog

level: log_debug

components:

libfuse:

attribute-expiration-sec: 120

entry-expiration-sec: 120

negative-entry-expiration-sec: 240

file_cache:

path: /mnt/ramdisk/blobtestrhel

timeout-sec: 120

max-size-mb: 4096

attr_cache:

timeout-sec: 7200

azstorage:

type: block

account-name: blobtestrhel

sas:

endpoint: https://blobtestrhel.blob.core.windows.net/

mode: sas

container: data

souravgupta-msft commented 2 months ago

Can you specify the proxy in environment variable http_proxy or https_proxy and retry mounting.

thekoolestkat commented 2 months ago

Hi Please see below: [xxxx-xxx@dvxxxxx002 ~]$ echo $http_proxy

http://xxx.xx.xx.69:8080/

[xxxx-xxx@dvxxxxx002 ~]$ echo $https_proxy

http://xxx.xx.xx.69:8080/

[xxxx-xxx@dvxxxxx002 ~]$

souravgupta-msft commented 2 months ago

@thekoolestkat, can you retry mounting after setting either of the environment variable. Please share the debug logs if the mount is failing.

thekoolestkat commented 2 months ago

Hi thanks for the reply I have had the environment variables set to the proxy. Can you confirm what log files are you after ?/var/log/blobfuse2.log Thanks,

souravgupta-msft commented 2 months ago

Yes, please share the logs in /var/log/blobfuse2.log

thekoolestkat commented 2 months ago

Hi please find attached the log file blobfuse2.log

souravgupta-msft commented 2 months ago

proxyconnect tcp: dial tcp: lookup https://xxx.xx.xxx.42: no such host Is the proxy configured correctly? Can you test the connectivity using wget or curl command? Also please update the blobfuse2 version to 2.3.0.

thekoolestkat commented 2 months ago

Hi Thanks for getting back to me so I tried few things below: Curl Prior to upgrade with No Porxy server CURL prior to upgradeWithProxyServer

Curl prior to upgrade with a proxy server configured CurlPriorToUpgrade

Mount Prior to upgrade no proxy server configured Mount prior to upgrade No Proxy Servers configured

Mount Prior to upgrade with a proxy server configured Mount prior to upgrade with Proxy servers

Upgraded to 2.3.0 mount with proxy still fails with Mount Upgraded to 2 3 0 – Mount still failed

Blobfuse2.log file attached

blobfuse2-22-5-24.log

souravgupta-msft commented 1 month ago

@thekoolestkat, sorry for replying late. I was able to setup proxy and replicate this error. It is a bug in the code, and we will fix this in our next release. For now, as temporary mitigation, you can just set https_proxy as xxx.xx.xx.42 (without protocol).