search

Azure / azure-functions-core-tools

Command line tools for Azure Functions
MIT License
1.3k stars 431 forks source link

Azure.Storage.Blobs: Service request failed #2959

Open Sti2nd opened 2 years ago

Sti2nd commented 2 years ago

Running timer triggers on my local computer suddenly stopped working a few days ago. The full error message is:

The listener for function 'SomeTimerTrigger' was unable to start. Azure.Storage.Blobs: Service request failed

In the Service Dependencies output, we have the following output. I have anonymised some information.

``` Ensuring Azure Functions Core Tools are up to date. This may take a few minutes... Project: C:\PROGRAM FILES\MICROSOFT VISUAL STUDIO\2022\PROFESSIONAL\COMMON7\IDE\EXTENSIONS\MICROSOFT\Azure Storage Emulator\azurite.exe --location "C:\Users\USERNAME\AppData\Local\Temp\Azurite" --debug "C:\Users\USERNAME\AppData\Local\Temp\Azurite\debug.log" Azure Functions Core Tools are up to date. Project: Azurite Blob service is starting at http://127.0.0.1:10000 Project: Azurite Blob service is successfully listening at http://127.0.0.1:10000 Project: Azurite Queue service is starting at http://127.0.0.1:10001 Project: Azurite Queue service is successfully listening at http://127.0.0.1:10001 Project: Azurite Table service is starting at http://127.0.0.1:10002 Project: Azurite Table service is successfully listening at http://127.0.0.1:10002 Project: 127.0.0.1 - - [22/Feb/2022:10:43:02 +0000] "PUT /devstoreaccount1/testhubname-applease?restype=container HTTP/1.1" 409 - Project: 127.0.0.1 - - [22/Feb/2022:10:43:02 +0000] "PUT /devstoreaccount1/testhubname-applease/testhubname-appleaseinfo HTTP/1.1" 409 - Project: 127.0.0.1 - - [22/Feb/2022:10:43:02 +0000] "PUT /devstoreaccount1/testhubname-leases?restype=container HTTP/1.1" 409 - Project: 127.0.0.1 - - [22/Feb/2022:10:43:02 +0000] "PUT /devstoreaccount1/testhubname-leases?restype=container HTTP/1.1" 409 - Project: 127.0.0.1 - - [22/Feb/2022:10:43:02 +0000] "HEAD /devstoreaccount1/testhubname-leases/taskhub.json HTTP/1.1" 200 91 Project: 127.0.0.1 - - [22/Feb/2022:10:43:02 +0000] "HEAD /devstoreaccount1/testhubname-leases/taskhub.json HTTP/1.1" 200 91 Project: 127.0.0.1 - - [22/Feb/2022:10:43:02 +0000] "HEAD /devstoreaccount1/testhubname-leases/taskhub.json HTTP/1.1" 200 91 Project: 127.0.0.1 - - [22/Feb/2022:10:43:02 +0000] "HEAD /devstoreaccount1/testhubname-leases/taskhub.json HTTP/1.1" 200 91 Project: 127.0.0.1 - - [22/Feb/2022:10:43:02 +0000] "GET /devstoreaccount1/testhubname-leases/taskhub.json HTTP/1.1" 200 91 Project: 127.0.0.1 - - [22/Feb/2022:10:43:02 +0000] "GET /devstoreaccount1/testhubname-leases/taskhub.json HTTP/1.1" 200 91 Project: 127.0.0.1 - - [22/Feb/2022:10:43:02 +0000] "PUT /devstoreaccount1/azure-webjobs-hosts/locks/NODENAME-1124157677/Project.Functions.TimerTrigger.Some.TimerTrigger.Listener?comp=lease HTTP/1.1" 201 - Project: 127.0.0.1 - - [22/Feb/2022:10:43:02 +0000] "POST /devstoreaccount1/Tables() HTTP/1.1" 409 - Project: 127.0.0.1 - - [22/Feb/2022:10:43:02 +0000] "POST /devstoreaccount1/Tables() HTTP/1.1" 409 - Project: 127.0.0.1 - - [22/Feb/2022:10:43:02 +0000] "PUT /devstoreaccount1/testhubname-workitems HTTP/1.1" 204 - Project: 127.0.0.1 - - [22/Feb/2022:10:43:02 +0000] "PUT /devstoreaccount1/testhubname-control-01 HTTP/1.1" 204 - Project: 127.0.0.1 - - [22/Feb/2022:10:43:02 +0000] "GET /devstoreaccount1/azure-webjobs-hosts/timers/NODENAME-1124157677/Project.Functions.TimerTrigger.Some.TimerTrigger/status HTTP/1.1" 500 129 ```

Details copied from the exception follows. I have removed the computer name.

``` Category: Host.Startup Client IP address: 0.0.0.0 Event time: 22/02/2022 10:43:30 Exception type: Microsoft.Azure.WebJobs.Host.Listeners.FunctionListenerException FormattedMessage: The listener for function 'SomeTimerTrigger' was unable to start. HostInstanceId: eb4b02e6-1f82-4f4a-a513-e2d031b58405 LogLevel: Error Message: The listener for function 'SomeTimerTrigger' was unable to start. Node name: (redacted by reporter @Sti2nd) Problem ID: Microsoft.Azure.WebJobs.Host.Listeners.FunctionListenerException at ProcessId: 10344 prop__{OriginalFormat}: The listener for function 'SomeTimerTrigger' was unable to start. Role instance: (redacted by reporter @Sti2nd) SDK version: azurefunctionscoretools: 3.0.3904 Severity level: Error Stack Trace Microsoft.Azure.WebJobs.Host.Listeners.FunctionListenerException: The listener for function 'SomeTimerTrigger' was unable to start. Azure.RequestFailedException: Service request failed. Status: 500 (Internal Server Error) Headers: Server: Azurite-Blob/3.14.1 ETag: "0x231D56C1EE9FE60" x-ms-blob-type: BlockBlob x-ms-lease-state: available x-ms-lease-status: unlocked x-ms-client-request-id: 07ca5baf-0716-466e-ba72-5cda5cd47141 x-ms-request-id: a9c19adf-c282-48b1-8689-9ec2a7add73a x-ms-version: 2020-10-02 Accept-Ranges: bytes Date: Tue, 22 Feb 2022 10:43:30 GMT x-ms-server-encrypted: true x-ms-blob-content-md5: /i0DJYHXwM7brV1Yy3Au2w== Connection: keep-alive Keep-Alive: REDACTED Last-Modified: Mon, 24 Jan 2022 14:37:16 GMT Content-Length: 129 Content-Type: application/octet-stream Content-MD5: /i0DJYHXwM7brV1Yy3Au2w== at Azure.Storage.Blobs.BlobRestClient+d__6.MoveNext at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter.GetResult at Azure.Storage.Blobs.Specialized.BlobBaseClient+d__57.MoveNext at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter.GetResult at Azure.Storage.Blobs.Specialized.BlobBaseClient+d__56.MoveNext at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter.GetResult at Azure.Storage.Blobs.Specialized.BlobBaseClient+d__53.MoveNext at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter.GetResult at Azure.Storage.Blobs.Specialized.BlobBaseClient+d__52.MoveNext at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter.GetResult at Azure.Storage.Blobs.Specialized.BlobBaseClient+d__50.MoveNext at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1+ConfiguredTaskAwaiter.GetResult at Azure.Storage.Blobs.Specialized.BlobBaseClient+d__48.MoveNext at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult at Microsoft.Azure.WebJobs.Extensions.Timers.AzureStorageScheduleMonitor+d__11.MoveNext at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult at Microsoft.Azure.WebJobs.Extensions.Timers.Listeners.TimerListener+d__31.MoveNext at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification at Microsoft.Azure.WebJobs.Host.Listeners.SingletonListener+d__13.MoveNext at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification at Microsoft.Azure.WebJobs.Host.Listeners.FunctionListener+d__13.MoveNext ```
michaelpeng36 commented 2 years ago

@fabiocav, would you be able to help with this? It looks like this issue is related to the TimerTrigger listener.

mvpindev commented 2 years ago

+1 Same issue on my end, with no changes.

Sti2nd commented 2 years ago

Would be great to get some help with this. Impossible to develop Durable Azure Functions for me now and they were already hard to do from before.

anthonychu commented 2 years ago

+ @vijayrkn in case this issue only affects VS.

vijayrkn commented 2 years ago

Based on the error message, I would be surprised if it only impacts Visual Studio.

One thing to try here might be to update the version of Azurite and see if it handles the scenario better. Based on the log above, the version of Azurite running is 3.14.1

https://github.com/Azure/Azurite/releases/tag/v3.16.0 3.16 is the latest version available and VS is updated to ship this version in the next release. in the meantime, can you try installing the Azurite from Node globally on the machine with the latest version. Close VS, run azurite manually and re-open VS?

This should confirm if this is an issue on the Azurite side and if it is already fixed. Let me know this doesn't work.

Sti2nd commented 2 years ago

@vijayrkn Nice! I can confirm that it works with Azurite 3.16.0.

Which Visual Studio release will it ship in? I just got notified about and updated to version 17.1.1 of VS today; that still has Azurite 3.14.1.

vijayrkn commented 2 years ago

Glad to know it worked. We will be updating it in 17.2. A preview version of this release should be available soon.

ADH-LukeBollam commented 2 years ago

I just hit this today, with no code changes on my end. Confirmed that running Azurite outside of VS has fixed it, but updating the version in VS should really be high priority, even if its just pushing it to a minor patch.

Ideally we have the option to set the Azurite version in VS ourselves?

SandeepMattoo commented 2 years ago

Just for everyone's info.

I am facing same issue.

I was on VS17.1.1 - just updated to VS17.1.6. No difference, I still get the error.

will try updating Azurite :)

Sti2nd commented 2 years ago

Sad to report that there is no improvement in Visual Studio 17.2.1. Did you not release new Azurite with that version like you said, @vijayrkn?

image

Edit: Output from the VS installed azurite says it's the same version I have installed through npm. So I guess there must be something else wrong.

$ C:/'Program Files'/'Microsoft Visual Studio'/2022/Professional/Common7/IDE/Extensions/Microsoft/'Azure Storage Emulator'/azurite.exe --v
3.16.0
SignorLuigi commented 2 years ago

I started to have the same problem, until yesterday everything was working well, now all the sudden without any changes the Function just won't work. Visual Studio Version 17.2.1.

image

Any help would be appreciated. Thanks

Sti2nd commented 2 years ago

@mrnewrochelle This is a workaround https://github.com/Azure/azure-functions-core-tools/issues/2959#issuecomment-1059322454

SignorLuigi commented 2 years ago

@mrnewrochelle This is a workaround #2959 (comment)

Thanks but that it didn't work for me, I tried before, and I have the last version already.

SignorLuigi commented 2 years ago

Ok I found a workaround for those who didn't make it work with the previews one. Go to Azure Explorer Storage, search for the (Emulator - Default Ports) (Key) and clean up the containers, and that's all. I worked good for me.

image

briandunnington commented 2 years ago

Just wanted to confirm that @mrnewrochelle's suggestion worked for me. I had already tried installing gobally via npm and had the latest version (both via npm and VS) but was still facing errors. Deleting all of the containers from the emulator cleared it up for me.

JimiSweden commented 1 year ago

Ok I found a workaround for those who didn't get it working with the preview. Go to Azure Explorer Storage, search for the (Emulator - Default Ports) (Key) and clean up the containers, and that's all. It worked good for me.

image

to clarify:

  1. download "Azure Storage Explorer" https://azure.microsoft.com/en-us/products/storage/storage-explorer/#overview
  2. Delete what is in "azure-webjobs-hosts"
image
bhoukom commented 1 year ago

I installed the latest version of VS 2022 (17.4.4) and the issue was resolved.

khertwork commented 1 year ago

Using VS 2022 (17.4.4) and the issue still exist.

Asier-Villanueva commented 1 year ago

Ok I found a workaround for those who didn't make it work with the previews one. Go to Azure Explorer Storage, search for the (Emulator - Default Ports) (Key) and clean up the containers, and that's all. I worked good for me.

image

This worked for me.

Lylelljr commented 9 months ago

It's been a while since this thread has had an update.

This issue was resolved for me by updating Visual Studio 2022 to 17.8.3.