search

Azure / Azurite

A lightweight server clone of Azure Storage that simulates most of the commands supported by it with minimal dependencies
MIT License
1.83k stars 325 forks source link

Azurite returns an error when `filter` is empty while querying table entities #1573

Open craxal opened 2 years ago

craxal commented 2 years ago

Which service(blob, file, queue, table) does this issue concern?

Table

Which version of the Azurite was used?

3.18.0

Where do you get Azurite? (npm, DockerHub, NuGet, Visual Studio Code Extension)

npm

What's the Node.js version?

16.14.2

What problem was encountered?

When an empty filter parameter is used to query entities, the following error is returned:

{"odata.error":{"code":"InvalidInput","message":{"lang":"en-US","value":"The query condition specified in the request is invalid.\nRequestId:9c789a8e-5989-41c7-8180-b853dee771ec\nTime:2022-06-27T22:01:15.449Z"}}}

Azurite outputs this as well:

127.0.0.1 - - [27/Jun/2022:22:01:15 +0000] "GET /devstoreaccount1/test03()?$select=&$filter= HTTP/1.1" 400 -

Public Azure does not return an error when the filter is empty.

Steps to reproduce the issue?

  1. Install and launch Azurite from the command line.
  2. Install and launch Storage Explorer.
  3. In Storage Explorer, create an emulator table and open it.
  4. Open the Query Builder and remove the default clauses.
  5. Run the query.

Have you found a mitigation/solution?

No

edwin-huber commented 2 years ago

Hi @craxal thanks for raising this. It looks like an edge case, can you confirm? It will help us prioritize. Thanks!

craxal commented 2 years ago

I would consider this an edge case, yes. Technically, it shouldn't be possible to have a no-clause query in Storage Explorer; we auto-generate a clause if that happens. That code wasn't working, which is how we found this bug.

pregress commented 4 weeks ago

We have the same issue using the health checks feature in asp.net 8. So I don't think this qualifies as an edge case anymore as this is also in use in .net aspire.

This call: https://github.com/Xabaril/AspNetCore.Diagnostics.HealthChecks/blob/116a4eac95eac47d1811d33a1f93b053ff87fa19/src/HealthChecks.Azure.Data.Tables/AzureTableServiceHealthCheck.cs#L37

I assume because they just use false as a filter expression. The actual azure storage implementation doesn't have an issue with this.

Azurite version: 3.28.0 Docker version: 26.1.4, build 5650f9b

export AZURITE_ACCOUNTS="azurite:{redacted}:{redacted};"
azurite --oauth basic -l /data --tableHost 0.0.0.0 -d /data/debug.log --cert /usr/local/share/ca-certificates/our.crt --key /usr/local/share/ca-certificates/our.key

This is the debug log:

2024-10-15T12:37:01.103Z 9c5ee86c-1701-4f68-8271-6893829c1ffc info: TableStorageContextMiddleware: RequestMethod=GET RequestURL=https://azurite.table.docker.local/Tables?$format=application%2Fjson%3Bodata%3Dminimalmetadata&$filter=false RequestHeaders:{"host":"azurite.table.docker.local:10002","x-ms-version":"2020-12-06","dataserviceversion":"3.0","accept":"application/json;odata=minimalmetadata","x-ms-client-request-id":"2d093992-dc4f-46ff-b5ad-279d199bdde0","x-ms-return-client-request-id":"true","user-agent":"azsdk-net-Data.Tables/12.9.1 (.NET 8.0.7; Debian GNU/Linux 12 (bookworm))","authorization":"Bearer REDACTED","traceparent":"00-6e313985a22f2a03d1321b54adead265-edf0850c668a0ec1-00"} ClientIP=172.25.0.4 Protocol=https HTTPVersion=1.1 2024-10-15T12:37:01.103Z 9c5ee86c-1701-4f68-8271-6893829c1ffc debug: tableStorageContextMiddleware: Dispatch pattern string: /Tables 2024-10-15T12:37:01.104Z 9c5ee86c-1701-4f68-8271-6893829c1ffc info: tableStorageContextMiddleware: Account=azurite tableName=undefined 2024-10-15T12:37:01.104Z 9c5ee86c-1701-4f68-8271-6893829c1ffc verbose: DispatchMiddleware: Dispatching request... 2024-10-15T12:37:01.104Z 9c5ee86c-1701-4f68-8271-6893829c1ffc info: DispatchMiddleware: Operation=Table_Query 2024-10-15T12:37:01.104Z 9c5ee86c-1701-4f68-8271-6893829c1ffc verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications. 2024-10-15T12:37:01.104Z 9c5ee86c-1701-4f68-8271-6893829c1ffc info: TableSharedKeyLiteAuthenticator:validate() Start validation against account shared key authentication. 2024-10-15T12:37:01.104Z 9c5ee86c-1701-4f68-8271-6893829c1ffc info: TableSharedKeyLiteAuthenticator:validate() Request doesn't include valid authentication header. Skip SharedKeyLite authentication. 2024-10-15T12:37:01.104Z 9c5ee86c-1701-4f68-8271-6893829c1ffc info: TableSharedKeyAuthenticator:validate() Start validation against account shared key authentication. 2024-10-15T12:37:01.105Z 9c5ee86c-1701-4f68-8271-6893829c1ffc info: TableSharedKeyAuthenticator:validate() [STRING TO SIGN]:"GET\n\n\n\n/azurite/Tables" 2024-10-15T12:37:01.105Z 9c5ee86c-1701-4f68-8271-6893829c1ffc info: TableSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey azurite:gIKgLCnSfRKOXnJGlGEIehzj36KyRyPi/plhLrAWdBo= 2024-10-15T12:37:01.105Z 9c5ee86c-1701-4f68-8271-6893829c1ffc info: TableSharedKeyAuthenticator:validate() Calculated authentication header based on key2: SharedKey azurite:6lN/Q8YTJzGra1kZS8DT7C0XYEY5duK6sjc+5wiyIjM= 2024-10-15T12:37:01.105Z 9c5ee86c-1701-4f68-8271-6893829c1ffc info: TableSharedKeyAuthenticator:validate() Validation failed. 2024-10-15T12:37:01.105Z 9c5ee86c-1701-4f68-8271-6893829c1ffc info: AccountSASAuthenticator:validate() Start validation against account Shared Access Signature pattern. 2024-10-15T12:37:01.105Z 9c5ee86c-1701-4f68-8271-6893829c1ffc debug: AccountSASAuthenticator:validate() Getting account properties... 2024-10-15T12:37:01.105Z 9c5ee86c-1701-4f68-8271-6893829c1ffc debug: AccountSASAuthenticator:validate() Retrieved account name from context: azurite, table: undefined 2024-10-15T12:37:01.105Z 9c5ee86c-1701-4f68-8271-6893829c1ffc debug: AccountSASAuthenticator:validate() Got account properties successfully. 2024-10-15T12:37:01.105Z 9c5ee86c-1701-4f68-8271-6893829c1ffc debug: AccountSASAuthenticator:validate() Retrieved signature from URL parameter sig: undefined 2024-10-15T12:37:01.105Z 9c5ee86c-1701-4f68-8271-6893829c1ffc info: AccountSASAuthenticator:validate() Failed to get valid account SAS values from request. 2024-10-15T12:37:01.105Z 9c5ee86c-1701-4f68-8271-6893829c1ffc info: TableSASAuthenticator:validate() Start validation against table service Shared Access Signature pattern. 2024-10-15T12:37:01.105Z 9c5ee86c-1701-4f68-8271-6893829c1ffc debug: TableSASAuthenticator:validate() Getting account properties... 2024-10-15T12:37:01.105Z 9c5ee86c-1701-4f68-8271-6893829c1ffc error: TableSASAuthenticator:validate() table name is undefined in context. 2024-10-15T12:37:01.105Z 9c5ee86c-1701-4f68-8271-6893829c1ffc info: TableTokenAuthenticator:validate() Start validation against token authentication. 2024-10-15T12:37:01.105Z 9c5ee86c-1701-4f68-8271-6893829c1ffc info: TableTokenAuthenticator:authenticateBasic() Validation against token authentication successfully. 2024-10-15T12:37:01.106Z 9c5ee86c-1701-4f68-8271-6893829c1ffc verbose: DeserializerMiddleware: Start deserializing... 2024-10-15T12:37:01.106Z 9c5ee86c-1701-4f68-8271-6893829c1ffc info: HandlerMiddleware: DeserializedParameters={"options":{"queryOptions":{"format":"application/json;odata=minimalmetadata","filter":"false"},"requestId":"2d093992-dc4f-46ff-b5ad-279d199bdde0","dataServiceVersion":"3.0"},"version":"2020-12-06"} 2024-10-15T12:37:01.108Z 9c5ee86c-1701-4f68-8271-6893829c1ffc error: ErrorMiddleware: Received a MiddlewareError, fill error information to HTTP response 2024-10-15T12:37:01.109Z 9c5ee86c-1701-4f68-8271-6893829c1ffc error: ErrorMiddleware: ErrorName=StorageError ErrorMessage="The query condition specified in the request is invalid." ErrorHTTPStatusCode=400 ErrorHTTPStatusMessage=undefined ErrorHTTPHeaders={"x-ms-error-code":"InvalidInput","x-ms-request-id":"9c5ee86c-1701-4f68-8271-6893829c1ffc","x-ms-version":"2023-11-03"} ErrorHTTPBody="{\"odata.error\":{\"code\":\"InvalidInput\",\"message\":{\"lang\":\"en-US\",\"value\":\"The query condition specified in the request is invalid.\nRequestId:9c5ee86c-1701-4f68-8271-6893829c1ffc\nTime:2024-10-15T12:37:01.108Z\"}}}" ErrorStack="\"StorageError: The query condition specified in the request is invalid.\n at Function.getQueryConditionInvalid (/opt/azurite/dist/src/table/errors/StorageErrorFactory.js:83:16)\n at LokiTableMetadataStore.queryTable (/opt/azurite/dist/src/table/persistence/LokiTableMetadataStore.js:179:49)\n at TableHandler.query (/opt/azurite/dist/src/table/handlers/TableHandler.js:74:71)\n at /opt/azurite/dist/src/table/generated/middleware/HandlerMiddlewareFactory.js:58:18\n at /opt/azurite/dist/src/table/generated/ExpressMiddlewareFactory.js:77:63\n at Layer.handle [as handle_request] (/opt/azurite/node_modules/express/lib/router/layer.js:95:5)\n at trim_prefix (/opt/azurite/node_modules/express/lib/router/index.js:328:13)\n at /opt/azurite/node_modules/express/lib/router/index.js:286:9\n at Function.process_params (/opt/azurite/node_modules/express/lib/router/index.js:346:12)\n at next (/opt/azurite/node_modules/express/lib/router/index.js:280:10)\"" 2024-10-15T12:37:01.109Z 9c5ee86c-1701-4f68-8271-6893829c1ffc error: ErrorMiddleware: Set HTTP code: 400 2024-10-15T12:37:01.109Z 9c5ee86c-1701-4f68-8271-6893829c1ffc error: ErrorMiddleware: Set HTTP Header: x-ms-error-code=InvalidInput 2024-10-15T12:37:01.109Z 9c5ee86c-1701-4f68-8271-6893829c1ffc error: ErrorMiddleware: Set HTTP Header: x-ms-request-id=9c5ee86c-1701-4f68-8271-6893829c1ffc 2024-10-15T12:37:01.109Z 9c5ee86c-1701-4f68-8271-6893829c1ffc error: ErrorMiddleware: Set HTTP Header: x-ms-version=2023-11-03 2024-10-15T12:37:01.109Z 9c5ee86c-1701-4f68-8271-6893829c1ffc error: ErrorMiddleware: Set content type: application/json;odata=minimalmetadata;streaming=true;charset=utf-8 2024-10-15T12:37:01.109Z 9c5ee86c-1701-4f68-8271-6893829c1ffc error: ErrorMiddleware: Set HTTP body: "{\"odata.error\":{\"code\":\"InvalidInput\",\"message\":{\"lang\":\"en-US\",\"value\":\"The query condition specified in the request is invalid.\nRequestId:9c5ee86c-1701-4f68-8271-6893829c1ffc\nTime:2024-10-15T12:37:01.108Z\"}}}" 2024-10-15T12:37:01.110Z 9c5ee86c-1701-4f68-8271-6893829c1ffc info: EndMiddleware: End response. TotalTimeInMS=7 StatusCode=400 StatusMessage=Bad Request Headers={"server":"Azurite-Table/3.28.0","x-ms-error-code":"InvalidInput","x-ms-request-id":"9c5ee86c-1701-4f68-8271-6893829c1ffc","x-ms-version":"2023-11-03","content-type":"application/json;odata=minimalmetadata;streaming=true;charset=utf-8"}