Datastore emulator with arg --use-firestore-in-datastore-mode fails when multiple values are passed as Array with IN filter

[onurdialpad]

[REQUIRED] Environment info

firebase-tools: 13.11.3

Platform: macOS

[REQUIRED] Test case

import requests

#emulator_command = gcloud beta emulators datastore start --no-store-on-disk --host-port=127.0.0.1:54510  --use-firestore-in-datastore-mode

emulator_url = 'http://127.0.0.1:54510/v1/projects/testbed-test:runQuery'

payload = b'{"partitionId": {"projectId": "testbed-test", "namespaceId": ""}, "query": {"kind": [{"name": "QueryTestEntity"}], "filter": {"propertyFilter": {"op": "IN", "property": {"name": "str_prop"}, "value": {"excludeFromIndexes": false, "arrayValue": {"values": [{"excludeFromIndexes": false, "stringValue": "foo"}, {"excludeFromIndexes": false, "stringValue": "bar"}]}}}}, "limit": 100}, "readOptions": {"readConsistency": "EVENTUAL"}}'

headers = {'Content-Length': '429', 'Content-Type': 'application/json'}

# POST request to the emulator
resp = requests.post(emulator_url, headers=headers, data=payload)

print(resp.json())
#  {"error":{"code":400,"message":"Filter has 2 properties, expected 1","status":"INVALID_ARGUMENT"}}

[REQUIRED] Steps to reproduce

1 - run the datastore emulator as

gcloud beta emulators datastore start --no-store-on-disk --host-port=127.0.0.1:54510  --use-firestore-in-datastore-mode

2 - Execute the test code shared above

[REQUIRED] Expected behavior

firestore emulator in datastore mode returns the response below when I execute it with a command:

gcloud emulators firestore start --database-mode=datastore-mode --host-port=127.0.0.1:54510

{'batch': {'entityResultType': 'FULL', 'moreResults': 'MORE_RESULTS_AFTER_LIMIT', 'readTime': '2024-06-25T19:44:40.725787Z'}} 

[REQUIRED] Actual behavior

It returns http status code 400 with error message

{"error":{"code":400,"message":"Filter has 2 properties, expected 1","status":"INVALID_ARGUMENT"}}

[onurdialpad]

Note, datastore emulator with arg --use-firestore-in-datastore-mode works when I post IN filter with single element, payload is

{'kind': [{'name': 'QueryTestEntity'}], 'filter': {'propertyFilter': {'op': 'IN', 'property': {'name': 'str_prop'}, 'value': {'excludeFromIndexes': False, 'arrayValue': {'values': [{'excludeFromIndexes': False, 'stringValue': 'foo'}]}}}}, 'limit': 100}

[harshyyy21]

Hi @onurdialpad, the Datastore emulator does not support the new query features even for datastore mode, hence why you see that exception.

[onurdialpad]

Hi @harshyyy21, thanks for the answer. I thought It is supposed to work with IN filter, since It works with or without --use-firestore-in-datastore-mode when I pass

{'kind': [{'name': 'QueryTestEntity'}], 'filter': {'propertyFilter': {'op': 'IN', 'property': {'name': 'str_prop'}, 'value': {'excludeFromIndexes': False, 'arrayValue': {'values': [{'excludeFromIndexes': False, 'stringValue': 'foo'}]}}}}, 'limit': 100}

to datastore emulator. As far as I know IN filter is something new Firestore supports. Am I missing something?

[harshyyy21]

When IN was added to Datastore Mode, it was never backfilled into the Datastore Emulator. The query above is essentially an equality with a single value, but as soon as you add multiple values in the array, it will fail with or without the --use-in-firestore-in-datastore-mode flag. IN along with the other new query features are only available via the Firestore emulator.

[onurdialpad]

Thanks, closing this then.