Lists broken: "contains unrecognized parameter: [ttl]"

[netdever]

When I try to reference a list in my ES query, I receive the below errors:

[Wed Jan 02 16:17:57.967308 2019] [php7:notice] [pid 7370] [client IPADDRESS:PORT] except [NONE] FOO\SearchException: "{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"request [/lookup_tables/esq_lookup/1546463877_1589251599] contains unrecognized parameter: [ttl]"}],"type":"illegal_argument_exception","reason":"request [/lookup_tables/esq_lookup/1546463877_1589251599] contains unrecognized parameter: [ttl]"},"status":400}" at [/var/www/411/phplib/Search/Elasticsearch.php:311] 0:[FOO\Elasticsearch_Search->search() called at [/var/www/411/phplib/Search/Elasticsearch.php:140]] 1:[FOO\Elasticsearch_Search->_execute() called at [/var/www/411/phplib/Search.php:273]] 2:[FOO\Search->execute() called at [/var/www/411/phplib/Job/Search.php:58]] 3:[FOO\Search_Job->_run() called at [/var/www/411/phplib/REST/Searches.php:212]] 4:[FOO\Searches_REST->test() called at [/var/www/411/phplib/REST/Searches.php:60]] 5:[FOO\Searches_REST->POST() called at [/var/www/411/phplib/REST.php:108]] 6:[FOO\REST->route() called at [/var/www/411/htdocs/api/searches.php:6]], referer: https://fouroneone.mydomain.com/searches/new?type=es [Wed Jan 02 16:17:57.976394 2019] [php7:notice] [pid 7370] [client IPADDRESS:PORT] err [JOB] Search error id:[] job_id:[] ignorable:[] errors:[0:[SearchException: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"request [/lookup_tables/esq_lookup/1546463877_1589251599] contains unrecognized parameter: [ttl]"}],"type":"illegal_argument_exception","reason":"request [/lookup_tables/esq_lookup/1546463877_1589251599] contains unrecognized parameter: [ttl]"},"status":400}]], referer: https://fouroneone.mydomain.com/searches/new?type=es

[ceeeekay]

+1 for this, also if I use (@listname) in parentheses, and try to use a line- or comma-separated lists, I get Catch all: Invalid list and the following in my web logs:

[Tue Jan 22 14:40:34.278005 2019] [:error] [pid 11453] [client 192.168.2.72:35670] except [NONE] ESQuery\\SyntaxError: "
Invalid list" at [/var/www/411/vendor/kiwiz/esquery/src/InternalParser.php:240] 0:[ESQuery\\InternalParser->peg_buildExc
eption() called at [/var/www/411/vendor/kiwiz/esquery/src/InternalParser.php:131]] 1:[ESQuery\\InternalParser->error() c
alled at [/var/www/411/vendor/kiwiz/esquery/src/InternalParser.php:4159]] 2:[ESQuery\\InternalParser->ESQuery\\{closure}
() called at [:0]] 3:[call_user_func() called at [/var/www/411/vendor/kiwiz/esquery/src/InternalParser.php:3277]] 4:[ESQ
uery\\InternalParser->peg_parseQueryClause() called at [/var/www/411/vendor/kiwiz/esquery/src/InternalParser.php:2798]]
5:[ESQuery\\InternalParser->peg_parseQueryUnaryNOT() called at [/var/www/411/vendor/kiwiz/esquery/src/InternalParser.php
:2633]] 6:[ESQuery\\InternalParser->peg_parseQueryNOT() called at [/var/www/411/vendor/kiwiz/esquery/src/InternalParser.
php:2520]] 7:[ESQuery\\InternalParser->peg_parseQueryAND() called at [/var/www/411/vendor/kiwiz/esquery/src/InternalPars
er.php:2429]] 8:[ESQuery\\InternalParser->peg_parseQueryOR() called at [/var/www/411/vendor/kiwiz/esquery/src/InternalPa
rser.php:2401]] 9:[ESQuery\\InternalParser->peg_parseQueryCommand() called at [/var/www/411/vendor/kiwiz/esquery/src/Int
ernalParser.php:482]] 10:[ESQuery\\InternalParser->peg_parseRoot() called at [:0]] 11:[call_user_func() called at [/var/
www/411/vendor/kiwiz/esquery/src/InternalParser.php:4223]] 12:[ESQuery\\InternalParser->parse() called at [/var/www/411/
phplib/Search/Elasticsearch.php:91]] 13:[FOO\\Elasticsearch_Search->constructQuery() called at [/var/www/411/phplib/Sear
ch.php:272]] 14:[FOO\\Search->execute() called at [/var/www/411/phplib/Job/Search.php:58]] 15:[FOO\\Search_Job->_run() c
alled at [/var/www/411/phplib/REST/Searches.php:212]] 16:[FOO\\Searches_REST->test() called at [/var/www/411/phplib/REST
/Searches.php:60]] 17:[FOO\\Searches_REST->POST() called at [/var/www/411/phplib/REST.php:108]] 18:[FOO\\REST->route() c
alled at [/var/www/411/htdocs/api/searches.php:6]], referer: https://411.domain.com/searches/new?type=es
[Tue Jan 22 14:40:34.345678 2019] [:error] [pid 11453] [client 192.168.2.72:35670] err [JOB] Search error id:[] job_id:[
] ignorable:[] errors:[0:[Catch all: Invalid list]], referer: https://411.domain.com/searches/new?type=es

The list config is reporting Valid, and the correct count for the list.

Note that my list consists of FQDNs, which contain dashes and dots, and I'm unsure if these need escaping.

I attempted to use a JSON list to see if quoted values made any difference, but as there's no example JSON in the docs I can't get past Invalid data from url

e.g., {"list":["prod-web1.domain.com","prod-web2.domain.com"]}

411: 1.5.0 ES: 6.5.4

[dsvetlov]

I investigated similar problem with JOIN operator for searched. It rises same exception. My journey leads me to the problem that ttl was a possible parameter few years ago when you create documents. Leter it was deprecated and but stil was acceptable by ES. Nowdays ES doesn't accept it and return error. I'm not sure where it parameter uess but started from ull request to official php client here. https://github.com/elastic/elasticsearch-php/pull/894

[dsvetlov]

So finally I found solution of my problem. You can find my PR here https://github.com/kiwiz/esquery/pull/2 You need to rebuild your container with this updated ESQuery lib.

[gromit6891]

@dsvetlov Good find. How can the container be rebuilt with this ESQuery lib?

[dsvetlov]

Below you can find my repo with fixed docker and composer files. https://github.com/dsvetlov/411