infinilabs / gateway

🚀 A High-performance gateway designed for search scenarios. Good friend of Elasticsearch/Opensearch/Easysearch.
https://www.infinilabs.com/en/docs/latest/gateway/
28 stars 3 forks source link

no support _builk {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"}],"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"},"status":400} #39

Closed dyrnq closed 5 months ago

dyrnq commented 7 months ago

Description

{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"}],"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"},"status":400}

Steps to reproduce

  1. Start INFINI Gateway
  2. Second step
#!/usr/bin/env bash

# original es
es_host="http://192.168.6.175:9200"
# gateway es
es_host="http://192.168.6.177:8000"

curl \
-X POST \
-u elastic:test123 \
-H 'Content-Type: application/x-ndjson' \
${es_host}/_bulk \
-d '
{ "create" : { "_index": "test1" } }
{ "color": "create black" }
{ "create" : { "_index": "test2"} }
{ "color": "create black" }
'
sleep 3s;
curl -X GET -u elastic:test123 -H 'Content-Type: application/json' ${es_host}/test1/_search | jq

Test separately

case 1 sucess

# original es
es_host="http://192.168.6.175:9200"

case 2 no success

# gateway es
es_host="http://192.168.6.177:8000"
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"}],"type":"illegal_argument_exception","reason":"Action/metadata line [1] contains an unknown parameter [_type]"},"status":400}  

Expected behavior

{"errors":false,"took":11,"items":[{"create":{"_index":"test1","_id":"t_-nEY0Bq-FVL_Pe6TkL","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":4,"_primary_term":1,"status":201}},{"create":{"_index":"test2","_id":"uP-nEY0Bq-FVL_Pe6TkL","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":4,"_primary_term":1,"status":201}}]} 

Actual behavior

As mentioned above

Environment

es version

elasticsearch:8.11.3

[GATEWAY] A light-weight, powerful and high-performance search gateway. [GATEWAY] 1.21.0#1229, 2023-12-28 08:22:36, 2024-12-31 10:10:10, 5ca75b9539303a3ceb62e37e72811b580e7e004f gateway 1.21.0 1229 2023-12-28 08:22:36 +0000 UTC 2024-12-31 10:10:10 +0000 UTC 5ca75b9539303a3ceb62e37e72811b580e7e004f


* Operating system and version: 
```bash
NAME="Ubuntu"
VERSION="20.04.5 LTS (Focal Fossa)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 20.04.5 LTS"
VERSION_ID="20.04"
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
VERSION_CODENAME=focal
UBUNTU_CODENAME=focal
medcl commented 7 months ago

this is due to es 8 was removed the _type field in bulk metadata, there is some config we can do to avoid this situration, maybe you can paste your currently gateway config here.

dyrnq commented 7 months ago

this is due to es 8 was removed the _type field in bulk metadata, there is some config we can do to avoid this situration, maybe you can paste your currently gateway config here.

I copy past use this file https://raw.githubusercontent.com/infinilabs/testing/main/setup/gateway/cases/replication/replication_via-disk.yml

then modify some env configs

BACKUP_ENDPOINTS
PRIMARY_ENDPOINTS
LOGGING_ENDPOINTS
medcl commented 7 months ago

Modify bulk_request_mutate filter, set fix_null_type to false

image
dyrnq commented 7 months ago

tks reply, I will try.

medcl commented 5 months ago

fell free to reopen if not resolved.