Azure / Azurite

A lightweight server clone of Azure Storage that simulates most of the commands supported by it with minimal dependencies
MIT License
1.81k stars 321 forks source link

Logic App Standard vscode extension not able to get data from Table Service v3.25 #2082

Closed ccastrotrejo closed 1 year ago

ccastrotrejo commented 1 year ago

Which service(blob, file, queue, table) does this issue concern?

Table Service

Which version of the Azurite was used?

3.25.0

Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)

Visual Studio Code Extension

What's the Node.js version?

v18.2.0

What problem was encountered?

In the Logic Apss Standard vscode extension we use Azurite vscode extension to store the "runs" in the Table Service. We are able to store that data in it but we haven't been successful with gathering the data. We use the following API to get the data if it helps: GET /runtime/webhooks/workflow/api/management/workflows/testWorkflow/runs?api-version=2019-10-01-edge-preview

Github issue on our side for reference https://github.com/Azure/logicapps/issues/841#issuecomment-1650195926 StackOverflow issue for reference https://stackoverflow.com/questions/76758903/vs-code-has-suddenly-stopped-displaying-logic-app-standard-run-history

Steps to reproduce the issue?

To reproduce the issue you can follow the steps in here: https://learn.microsoft.com/en-us/azure/logic-apps/create-single-tenant-workflows-visual-studio-code#run-test-and-debug-locally

127.0.0.1 - - [28/Jul/2023:23:47:24 +0000] "GET /devstoreaccount1/flowf8c2cbf8781896eflows(PartitionKey='D5F93',RowKey='MYEDGEENVIRONMENT_FLOWLOOKUP-MYEDGERESOURCEGROUP-STATEFUL1')?&timeout=16&sv=2018-03-28&sig=*sanitized*&se=2025-07-28T23%3A45%3A41Z&srt=sco&ss=bqt&sp=racupwdl HTTP/1.1" 200 -

127.0.0.1 - - [28/Jul/2023:23:47:24 +0000] "GET /devstoreaccount1/flowf8c2cbf8781896ecu0040b4b1a2aa69a3eruns?$filter=%28%28%28PartitionKey%20eq%20%277A515%27%29%20and%20%28%28RowKey%20ge%20%274BA8E57EB1FE4BF8AFA650916ED32F27_%27%29%20and%20%28RowKey%20lt%20%274BA8E57EB1FE4BF8AFA650916ED32F27%60%27%29%29%29%20and%20%28DeletedTime%20eq%20datetime%271970-01-01T00%3A00%3A00.0000000Z%27%29%29%20and%20%28Status%20ne%20%27Deleted%27%29&&$top=30&timeout=16&sv=2018-03-28&sig=*sanitized*&se=2025-07-28T23%3A45%3A41Z&srt=sco&ss=bqt&sp=racupwdl HTTP/1.1" 200 -

127.0.0.1 - - [28/Jul/2023:23:53:55 +0000] "GET /devstoreaccount1/flowf8c2cbf8781896eflows(PartitionKey='D5F93',RowKey='MYEDGEENVIRONMENT_FLOWLOOKUP-MYEDGERESOURCEGROUP-STATEFUL1')?&timeout=16&sv=2018-03-28&sig=*sanitized*&se=2025-07-28T23%3A53%3A37Z&srt=sco&ss=bqt&sp=racupwdl HTTP/1.1" 200 -

127.0.0.1 - - [28/Jul/2023:23:53:55 +0000] "GET /devstoreaccount1/flowf8c2cbf8781896ecu0040b4b1a2aa69a3eruns?$filter=%28%28%28PartitionKey%20eq%20%277A515%27%29%20and%20%28%28RowKey%20ge%20%274BA8E57EB1FE4BF8AFA650916ED32F27_%27%29%20and%20%28RowKey%20lt%20%274BA8E57EB1FE4BF8AFA650916ED32F27%60%27%29%29%29%20and%20%28DeletedTime%20eq%20datetime%271970-01-01T00%3A00%3A00.0000000Z%27%29%29%20and%20%28Status%20ne%20%27Deleted%27%29&&$top=30&timeout=16&sv=2018-03-28&sig=*sanitized*&se=2025-07-28T23%3A53%3A37Z&srt=sco&ss=bqt&sp=racupwdl HTTP/1.1" 200 -

Have you found a mitigation/solution?

Going back to the 3.24.0 version Downgrading to the 3.24.0 indeed works and it also prompts the runs that were trigger in the 3.25.0 version.

garrettbaski commented 1 year ago

It looks like this may be same issue as #2069 since I see a check for DeletedTime eq datetime'1970-01-01T00:00:00.0000000Z. @notheotherben does that seem right?

ccastrotrejo commented 1 year ago

These are the errors that we are seeing from the terminal. @garrettbaski Please let me know if you need something specific from us

2023-07-31T17:00:43.350Z f4577019-c10a-4996-8764-aa21d45db950 error: ErrorMiddleware: Received a MiddlewareError, fill error information to HTTP response
2023-07-31T17:00:43.350Z f4577019-c10a-4996-8764-aa21d45db950 error: ErrorMiddleware: ErrorName=StorageError ErrorMessage="The table specified does not exist."  ErrorHTTPStatusCode=404 ErrorHTTPStatusMessage=undefined ErrorHTTPHeaders={"x-ms-error-code":"TableNotFound","x-ms-request-id":"f4577019-c10a-4996-8764-aa21d45db950","x-ms-version":"2023-01-03"} ErrorHTTPBody="{\"odata.error\":{\"code\":\"TableNotFound\",\"message\":{\"lang\":\"en-US\",\"value\":\"The table specified does not exist.\\nRequestId:f4577019-c10a-4996-8764-aa21d45db950\\nTime:2023-07-31T17:00:43.350Z\"}}}" ErrorStack="\"StorageError: The table specified does not exist.\\n    at Function.getTableNotExist (...path/.nvm/versions/node/v18.2.0/lib/node_modules/azurite/dist/src/table/errors/StorageErrorFactory.js:50:16)\\n    at LokiTableMetadataStore.getEntityCollection (...path/.nvm/versions/node/v18.2.0/lib/node_modules/azurite/dist/src/table/persistence/LokiTableMetadataStore.js:585:53)\\n    at LokiTableMetadataStore.queryTableEntities (...path/.nvm/versions/node/v18.2.0/lib/node_modules/azurite/dist/src/table/persistence/LokiTableMetadataStore.js:251:44)\\n    at TableHandler.queryEntities (...path/.nvm/versions/node/v18.2.0/lib/node_modules/azurite/dist/src/table/handlers/TableHandler.js:316:81)\\n    at ...path/.nvm/versions/node/v18.2.0/lib/node_modules/azurite/dist/src/table/generated/middleware/HandlerMiddlewareFactory.js:58:18\\n    at ...path/.nvm/versions/node/v18.2.0/lib/node_modules/azurite/dist/src/table/generated/ExpressMiddlewareFactory.js:77:63\\n    at Layer.handle [as handle_request] (...path/.nvm/versions/node/v18.2.0/lib/node_modules/azurite/node_modules/express/lib/router/layer.js:95:5)\\n    at trim_prefix (...path/.nvm/versions/node/v18.2.0/lib/node_modules/azurite/node_modules/express/lib/router/index.js:328:13)\\n    at ...path/.nvm/versions/node/v18.2.0/lib/node_modules/azurite/node_modules/express/lib/router/index.js:286:9\\n    at Function.process_params (...path/.nvm/versions/node/v18.2.0/lib/node_modules/azurite/node_modules/express/lib/router/index.js:346:12)\""
2023-07-31T17:00:43.350Z f4577019-c10a-4996-8764-aa21d45db950 error: ErrorMiddleware: Set HTTP code: 404
2023-07-31T17:00:43.350Z f4577019-c10a-4996-8764-aa21d45db950 error: ErrorMiddleware: Set HTTP Header: x-ms-error-code=TableNotFound
2023-07-31T17:00:43.350Z f4577019-c10a-4996-8764-aa21d45db950 error: ErrorMiddleware: Set HTTP Header: x-ms-request-id=f4577019-c10a-4996-8764-aa21d45db950
2023-07-31T17:00:43.350Z f4577019-c10a-4996-8764-aa21d45db950 error: ErrorMiddleware: Set HTTP Header: x-ms-version=2023-01-03
2023-07-31T17:00:43.350Z f4577019-c10a-4996-8764-aa21d45db950 error: ErrorMiddleware: Set content type: application/json;odata=minimalmetadata;streaming=true;charset=utf-8
2023-07-31T17:00:43.350Z f4577019-c10a-4996-8764-aa21d45db950 error: ErrorMiddleware: Set HTTP body: "{\"odata.error\":{\"code\":\"TableNotFound\",\"message\":{\"lang\":\"en-US\",\"value\":\"The table specified does not exist.\\nRequestId:f4577019-c10a-4996-8764-aa21d45db950\\nTime:2023-07-31T17:00:43.350Z\"}}}"
garrettbaski commented 1 year ago

Is this different from the above? In the original description it looks like Azurite was returning a 200 but giving back nothing is the response.

But this looks like Azurite is giving back a 404. Was this caused by the same query as in the original post or something else?

ccastrotrejo commented 1 year ago

Yes it was caused by the same interaction with azurite service. If it helps, I can send you an email with the azuritedebug.log file

garrettbaski commented 1 year ago

Oh, I notice that in my runs. I think when you create a new table the library just checks if it exists and then if not creates the table. So I think that is expected behavior and a red herring. I think for the error of no data being returned you are running in to the same as issue #2069. If I have some time I'll try and repro with the logic apps workflow you linked to, but no guarantees there. If you have time, you could try and pull #2077 and run with that version of azurite to double check if that solves the issue.

ccastrotrejo commented 1 year ago

Hi @garrettbaski, sure I can try it. Thanks!

ccastrotrejo commented 1 year ago

Hi @garrettbaski. This fixed it. Also, the newest 3.25.1 version in prod works fine for us. Thank you very much