search

manticoresoftware / manticoresearch

Easy to use open source fast database for search | Good alternative to Elasticsearch now | Drop-in replacement for E in the ELK soon
https://manticoresearch.com
GNU General Public License v3.0
9.04k stars 507 forks source link

Testing bug fixes in fuzzy search and autocomplete #2566

Closed PavelShilin89 closed 1 month ago

PavelShilin89 commented 2 months ago

Adjusted test-fuzzy-search-negative.rec, removed commands that did not generate errors when min_infix_len was enabled.

PavelShilin89 commented 2 months ago

@sanikolaev I need approval to merge

sanikolaev commented 1 month ago

"ERROR 1064 (42000) at line 1: Something went wrong" doesn't looks like normal behaviour? Did it crash? If so, it should be investigated and fixed.

sanikolaev commented 1 month ago

I also don't understand why here

mysql -h0 -P9306 -e "SELECT * FROM name WHERE MATCH('SMITH') OPTION fuzzy='0';"
––– output –––
ERROR 1064 (42000) at line 1: unknown option 'fuzzy'

The error says "unknown option 'fuzzy'" if this should be a known option.

PavelShilin89 commented 1 month ago

@dmitrykuzmenkov @sanikolaev discuss on the call tomorrow how to improve Something went wrong and unknown option fuzzy errors

sanikolaev commented 1 month ago

mysql -h0 -P9306 -e "SELECT * FROM name WHERE MATCH('SMITH') OPTION fuzzy='0';" ––– output ––– ERROR 1064 (42000) at line 1: unknown option 'fuzzy'

This is because the correct syntax is OPTION fuzzy=0. Let's change the error, so it looks like this:

ERROR 1064 (42000) at line 1: invalid value for option 'fuzzy'

ERROR 1064 (42000) at line 1: Something went wrong

This should be investigated too.

sanikolaev commented 1 month ago 
mysql -h0 -P9306 -e "CALL AUTOCOMPLETE('', 'name', 'us' AS layouts);"
––– output –––
ERROR 1064 (42000) at line 1: Something went wrong

should return an empty result instead (similarly to call keywords/suggest)

donhardman commented 1 month ago

Fixes are done here: https://github.com/manticoresoftware/manticoresearch-buddy/pull/355

We can proceed with validation after merge

github-actions[bot] commented 1 month ago

clt

👎 CLT tests in test/clt-tests/buddy/ ✅ OK: 3 ❌ Failed: 1 ⏳ Duration: 74s 👉 Check Action Results for commit bd24291 Failed tests:

test/clt-tests/buddy/test-fuzzy-autocomplete-http.rec ```diff ––– input ––– rm -f /var/log/manticore/searchd.log; searchd --stopwait > /dev/null; searchd; if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore/searchd.log;fi ––– output ––– Manticore %{SEMVER} %{COMMITDATE}#!/(\sdev)?\s/!#(columnar %{SEMVER} %{COMMITDATE}) (secondary %{SEMVER} %{COMMITDATE}) (knn %{SEMVER} %{COMMITDATE}) Copyright (c) 2001-2016, Andrew Aksyonoff Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com) Copyright (c) 2017-%{YEAR}, Manticore Software LTD (https://manticoresearch.com) [#!/[0-9]{2}:[0-9]{2}.[0-9]{3}/!#] [#!/[0-9]+/!#] using config file '/etc/manticoresearch/manticore.conf' (%{NUMBER} chars)... starting daemon version '%{SEMVER} %{COMMITDATE}#!/(\sdev)?\s/!#(columnar %{SEMVER} %{COMMITDATE}) (secondary %{SEMVER} %{COMMITDATE}) (knn %{SEMVER} %{COMMITDATE})' ... listening on %{IPADDR}:9312 for sphinx and http(s) listening on %{IPADDR}:9306 for mysql listening on %{IPADDR}:9308 for sphinx and http(s) Buddy started! ––– input ––– apt-get update -y > /dev/null; echo $? ––– output ––– 0 ––– input ––– apt-get install jq -y > /dev/null; echo $? ––– output ––– debconf: delaying package configuration, since apt-utils is not installed 0 ––– input ––– php -d memory_limit=-1 ./test/clt-tests/scripts/load_us_names.php 1000 10 1000000 5 2 > /dev/null ––– output ––– ––– input ––– mysql -h0 -P9306 -e "SHOW CREATE TABLE name;" | grep "min_infix_len='2'" | sed "s/.*\(min_infix_len='2'\).*/\1/" ––– output ––– min_infix_len='2' ––– input ––– mysql -h0 -P9306 -e "SELECT COUNT(*) FROM name;" ––– output ––– +----------+ | count(*) | +----------+ | 1000000 | +----------+ ––– input ––– curl -sX POST http://localhost:9308/search -d '{"index":"name","query":{"bool":{"must":[{"match":{"*":"RICH"}}]}},"options":{"fuzzy":true}}' | jq . ––– output ––– + { + "took": 4, + "timed_out": false, + "hits": { + "total": 8380, + "total_relation": "eq", + "hits": [ + { + "_id": 832342, + "_score": 1595, + "_source": { + "username": "RICO RICE" + } + }, + { + "_id": 768099, + "_score": 1562, + "_source": { + "username": "BRICE RICH" + } + }, + { + "_id": 989140, + "_score": 1549, + "_source": { + "username": "LEVI RICO" + } + }, + { + "_id": 869271, + "_score": 1549, + "_source": { + "username": "MARQUITA RICO" + } + }, + { + "_id": 932341, + "_score": 1549, + "_source": { + "username": "JERAMY RICO" + } + }, + { + "_id": 989530, + "_score": 1549, + "_source": { + "username": "RICO RAWLINGS" + } + }, + { + "_id": 939666, + "_score": 1549, + "_source": { + "username": "LAURENE RICO" + } + }, + { + "_id": 944740, + "_score": 1549, + "_source": { + "username": "JOSHUA RICO" + } + }, + { + "_id": 979783, + "_score": 1549, + "_source": { + "username": "RICO SWANK" + } + }, + { + "_id": 929967, + "_score": 1549, + "_source": { + "username": "BIANCA RICO" + } + }, + { + "_id": 914988, + "_score": 1549, + "_source": { + "username": "SUSANNE RICO" + } + }, + { + "_id": 911989, + "_score": 1549, + "_source": { + "username": "RICO IVY" + } + }, + { + "_id": 975060, + "_score": 1549, + "_source": { + "username": "RICO BRENNER" + } + }, + { + "_id": 921107, + "_score": 1549, + "_source": { + "username": "KENYETTA RICO" + } + }, + { + "_id": 996127, + "_score": 1549, + "_source": { + "username": "RICO WHITLOCK" + } + }, + { + "_id": 918295, + "_score": 1549, + "_source": { + "username": "MARLEN RICO" + } + }, + { + "_id": 933392, + "_score": 1549, + "_source": { + "username": "AN RICO" + } + }, + { + "_id": 924527, + "_score": 1549, + "_source": { + "username": "RICO BOEHM" + } + }, + { + "_id": 995638, + "_score": 1549, + "_source": { + "username": "RICO JAEGER" + } + }, + { + "_id": 950970, + "_score": 1549, + "_source": { + "username": "RICO TOTH" + } + } + ] + } + } ––– input ––– curl -sX POST http://localhost:9308/search -d '{"index":"name","query":{"bool":{"must":[{"match":{"*":"EBGRNA"}}]}},"options":{"fuzzy":true}}' | jq . ––– output ––– + { + "took": 3, + "timed_out": false, + "hits": { + "total": 4329, + "total_relation": "eq", + "hits": [ + { + "_id": 999307, + "_score": 1602, + "_source": { + "username": "BERNA WEIR" + } + }, + { + "_id": 929492, + "_score": 1602, + "_source": { + "username": "BERNA CHEUNG" + } + }, + { + "_id": 926530, + "_score": 1602, + "_source": { + "username": "BERNA GRAFF" + } + }, + { + "_id": 935711, + "_score": 1602, + "_source": { + "username": "BERNA PUGA" + } + }, + { + "_id": 985405, + "_score": 1602, + "_source": { + "username": "BERNA WOODWORTH" + } + }, + { + "_id": 943151, + "_score": 1602, + "_source": { + "username": "BERNA REICHERT" + } + }, + { + "_id": 916301, + "_score": 1602, + "_source": { + "username": "BERNA LATHAM" + } + }, + { + "_id": 928383, + "_score": 1602, + "_source": { + "username": "BERNA FRANZ" + } + }, + { + "_id": 868777, + "_score": 1602, + "_source": { + "username": "BERNA HAMLIN" + } + }, + { + "_id": 953801, + "_score": 1602, + "_source": { + "username": "BERNA POSTON" + } + }, + { + "_id": 936413, + "_score": 1602, + "_source": { + "username": "BERNA HOOPER" + } + }, + { + "_id": 987047, + "_score": 1602, + "_source": { + "username": "BERNA WITTE" + } + }, + { + "_id": 977094, + "_score": 1602, + "_source": { + "username": "BERNA ISBELL" + } + }, + { + "_id": 817074, + "_score": 1602, + "_source": { + "username": "BERNA VEGA" + } + }, + { + "_id": 799233, + "_score": 1602, + "_source": { + "username": "BERNA MOREY" + } + }, + { + "_id": 905588, + "_score": 1602, + "_source": { + "username": "BERNA GUILLORY" + } + }, + { + "_id": 798723, + "_score": 1602, + "_source": { + "username": "BERNA LOUIE" + } + }, + { + "_id": 788674, + "_score": 1602, + "_source": { + "username": "BERNA GILL" + } + }, + { + "_id": 790830, + "_score": 1602, + "_source": { + "username": "BERNA ABRAHAM" + } + }, + { + "_id": 889071, + "_score": 1602, + "_source": { + "username": "BERNA ELLIS" + } + } + ] + } + } ––– input ––– curl -sX POST http://localhost:9308/search -d '{"index":"name","query":{"bool":{"must":[{"match":{"*":"ANT"}}]}},"options":{"fuzzy":true}}' | jq . ––– output ––– + { + "took": 95, + "timed_out": false, + "hits": { + "total": 249348, + "total_relation": "eq", + "hits": [ + { + "_id": 855401, + "_score": 1538, + "_source": { + "username": "LUCIANA GANT" + } + }, + { + "_id": 796101, + "_score": 1538, + "_source": { + "username": "ANNIE GANT" + } + }, + { + "_id": 768944, + "_score": 1538, + "_source": { + "username": "ANNITA GANT" + } + }, + { + "_id": 773157, + "_score": 1536, + "_source": { + "username": "JOANNA DENT" + } + }, + { + "_id": 779895, + "_score": 1535, + "_source": { + "username": "ANNALISA KENT" + } + }, + { + "_id": 935071, + "_score": 1531, + "_source": { + "username": "DORIAN GANT" + } + }, + { + "_id": 929610, + "_score": 1531, + "_source": { + "username": "TANDRA GANT" + } + }, + { + "_id": 950473, + "_score": 1531, + "_source": { + "username": "MELANY GANT" + } + }, + { + "_id": 976868, + "_score": 1531, + "_source": { + "username": "STEFANIE GANT" + } + }, + { + "_id": 972411, + "_score": 1531, + "_source": { + "username": "JORDAN GANT" + } + }, + { + "_id": 895992, + "_score": 1531, + "_source": { + "username": "ANTONE GANT" + } + }, + { + "_id": 832469, + "_score": 1531, + "_source": { + "username": "STEFANIE GANT" + } + }, + { + "_id": 984473, + "_score": 1530, + "_source": { + "username": "LEONIE GANT" + } + }, + { + "_id": 959771, + "_score": 1530, + "_source": { + "username": "IRMA GANT" + } + }, + { + "_id": 917932, + "_score": 1530, + "_source": { + "username": "LISHA GANT" + } + }, + { + "_id": 970346, + "_score": 1530, + "_source": { + "username": "LORY GANT" + } + }, + { + "_id": 945404, + "_score": 1530, + "_source": { + "username": "EILENE GANT" + } + }, + { + "_id": 940539, + "_score": 1530, + "_source": { + "username": "NADINE GANT" + } + }, + { + "_id": 997179, + "_score": 1530, + "_source": { + "username": "DORSEY GANT" + } + }, + { + "_id": 934538, + "_score": 1530, + "_source": { + "username": "AUTUMN GANT" + } + } + ] + } + } ––– input ––– curl -sX POST http://localhost:9308/search -d '{"index":"name","query":{"bool":{"must":[{"match":{"*":"tony"}}]}},"options":{"fuzzy":true}}' | jq . ––– output ––– + { + "took": 4, + "timed_out": false, + "hits": { + "total": 4252, + "total_relation": "eq", + "hits": [ + { + "_id": 998183, + "_score": 1532, + "_source": { + "username": "TONA CHOATE" + } + }, + { + "_id": 999284, + "_score": 1532, + "_source": { + "username": "TONA EDMONDS" + } + }, + { + "_id": 932916, + "_score": 1532, + "_source": { + "username": "TONA GROGAN" + } + }, + { + "_id": 924415, + "_score": 1532, + "_source": { + "username": "TONA REYNOSO" + } + }, + { + "_id": 918798, + "_score": 1532, + "_source": { + "username": "TONA KOCH" + } + }, + { + "_id": 973217, + "_score": 1532, + "_source": { + "username": "TONA WINTER" + } + }, + { + "_id": 910284, + "_score": 1532, + "_source": { + "username": "TONA PRENTICE" + } + }, + { + "_id": 943295, + "_score": 1532, + "_source": { + "username": "TONA LAUGHLIN" + } + }, + { + "_id": 988528, + "_score": 1532, + "_source": { + "username": "TONA SUTHERLAND" + } + }, + { + "_id": 928754, + "_score": 1532, + "_source": { + "username": "TONA BARRETT" + } + }, + { + "_id": 916837, + "_score": 1532, + "_source": { + "username": "TONA IVORY" + } + }, + { + "_id": 993840, + "_score": 1532, + "_source": { + "username": "TONA MARQUARDT" + } + }, + { + "_id": 943890, + "_score": 1532, + "_source": { + "username": "TONA ASHE" + } + }, + { + "_id": 988892, + "_score": 1532, + "_source": { + "username": "TONA BOX" + } + }, + { + "_id": 943985, + "_score": 1532, + "_source": { + "username": "TONA DEHART" + } + }, + { + "_id": 971179, + "_score": 1532, + "_source": { + "username": "TONA KOZAK" + } + }, + { + "_id": 981673, + "_score": 1532, + "_source": { + "username": "TONA MCNEILL" + } + }, + { + "_id": 865983, + "_score": 1532, + "_source": { + "username": "TONA PETTIGREW" + } + }, + { + "_id": 982220, + "_score": 1532, + "_source": { + "username": "TONA ARNDT" + } + }, + { + "_id": 947393, + "_score": 1532, + "_source": { + "username": "TONA SEELEY" + } + } + ] + } + } ––– input ––– curl -sX POST http://localhost:9308/search -d '{"index":"name","query":{"bool":{"must":[{"match":{"*":"SMITH"}}]}}, "sort":[{"id":"desc"},{"username":"asc"}], "options":{"fuzzy":true}}' | jq . ––– output ––– + { + "took": 5, + "timed_out": false, + "hits": { + "total": 7170, + "total_relation": "eq", + "hits": [ + { + "_id": 999972, + "_score": 1, + "_source": { + "username": "KEITH SNOW" + } + }, + { + "_id": 999928, + "_score": 1, + "_source": { + "username": "BONITA LEONG" + } + }, + { + "_id": 999858, + "_score": 1, + "_source": { + "username": "TONITA GEARY" + } + }, + { + "_id": 999854, + "_score": 1, + "_source": { + "username": "DOROTHA WISEMAN" + } + }, + { + "_id": 999722, + "_score": 1, + "_source": { + "username": "SCOTT OLSON" + } + }, + { + "_id": 999721, + "_score": 1, + "_source": { + "username": "TONITA ESTRADA" + } + }, + { + "_id": 999263, + "_score": 1, + "_source": { + "username": "RUTHANN GREENE" + } + }, + { + "_id": 999249, + "_score": 1, + "_source": { + "username": "MINH LANDERS" + } + }, + { + "_id": 999126, + "_score": 1, + "_source": { + "username": "MALENA RUTHERFORD" + } + }, + { + "_id": 999119, + "_score": 1, + "_source": { + "username": "RUTHANNE CARDOSO" + } + }, + { + "_id": 998892, + "_score": 1, + "_source": { + "username": "IRWIN SOUTHARD" + } + }, + { + "_id": 998690, + "_score": 1, + "_source": { + "username": "MITCHELL SEITZ" + } + }, + { + "_id": 998580, + "_score": 1, + "_source": { + "username": "RAMONITA SWISHER" + } + }, + { + "_id": 998565, + "_score": 1, + "_source": { + "username": "RAMONITA MAYO" + } + }, + { + "_id": 998538, + "_score": 1, + "_source": { + "username": "ELSA SEITZ" + } + }, + { + "_id": 998525, + "_score": 1, + "_source": { + "username": "SCOTT CARRION" + } + }, + { + "_id": 997944, + "_score": 1, + "_source": { + "username": "RUTHE SCHENK" + } + }, + { + "_id": 997814, + "_score": 1, + "_source": { + "username": "MINH GOODWIN" + } + }, + { + "_id": 997587, + "_score": 1, + "_source": { + "username": "RAMONITA GIBBONS" + } + }, + { + "_id": 997557, + "_score": 1, + "_source": { + "username": "ARTIE SOUTH" + } + } + ] + } + } ––– input ––– curl -sX POST http://localhost:9308/search -d '{"index":"name","query":{"bool":{"must":[{"match":{"*":"SMYTH"}}]}}, "sort":[{"id":"desc"},{"username":"asc"}], "options":{"fuzzy":true}}' | jq . ––– output ––– + { + "took": 5, + "timed_out": false, + "hits": { + "total": 9965, + "total_relation": "eq", + "hits": [ + { + "_id": 999972, + "_score": 1, + "_source": { + "username": "KEITH SNOW" + } + }, + { + "_id": 999854, + "_score": 1, + "_source": { + "username": "DOROTHA WISEMAN" + } + }, + { + "_id": 999722, + "_score": 1, + "_source": { + "username": "SCOTT OLSON" + } + }, + { + "_id": 999498, + "_score": 1, + "_source": { + "username": "DORTHY DUCKWORTH" + } + }, + { + "_id": 999263, + "_score": 1, + "_source": { + "username": "RUTHANN GREENE" + } + }, + { + "_id": 999196, + "_score": 1, + "_source": { + "username": "HEATH MCNAMARA" + } + }, + { + "_id": 999126, + "_score": 1, + "_source": { + "username": "MALENA RUTHERFORD" + } + }, + { + "_id": 999119, + "_score": 1, + "_source": { + "username": "RUTHANNE CARDOSO" + } + }, + { + "_id": 999012, + "_score": 1, + "_source": { + "username": "HEATHER HAMMOCK" + } + }, + { + "_id": 998892, + "_score": 1, + "_source": { + "username": "IRWIN SOUTHARD" + } + }, + { + "_id": 998676, + "_score": 1, + "_source": { + "username": "BURT WOODWORTH" + } + }, + { + "_id": 998525, + "_score": 1, + "_source": { + "username": "SCOTT CARRION" + } + }, + { + "_id": 998220, + "_score": 1, + "_source": { + "username": "MORA WENTWORTH" + } + }, + { + "_id": 998083, + "_score": 1, + "_source": { + "username": "EDMUND ASHWORTH" + } + }, + { + "_id": 997999, + "_score": 1, + "_source": { + "username": "EDYTH CARLISLE" + } + }, + { + "_id": 997944, + "_score": 1, + "_source": { + "username": "RUTHE SCHENK" + } + }, + { + "_id": 997557, + "_score": 1, + "_source": { + "username": "ARTIE SOUTH" + } + }, + { + "_id": 997543, + "_score": 1, + "_source": { + "username": "HEATHER CLAUSEN" + } + }, + { + "_id": 997395, + "_score": 1, + "_source": { + "username": "JON ASHWORTH" + } + }, + { + "_id": 997286, + "_score": 1, + "_source": { + "username": "JOI FARNSWORTH" + } + } + ] + } + } ––– input ––– curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"jo","options":{"layouts":"us,ru"}}' | jq . ––– output ––– + [ + { + "total": 10, + "error": "", + "warning": "", + "columns": [ + { + "query": { + "type": "string" + } + } + ], + "data": [ + { + "query": "joe" + }, + { + "query": "jordan" + }, + { + "query": "joseph" + }, + { + "query": "john" + }, + { + "query": "jose" + }, + { + "query": "joy" + }, + { + "query": "joyce" + }, + { + "query": "jonas" + }, + { + "query": "johnson" + }, + { + "query": "major" + } + ] + } + ] ––– input ––– curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"jo","options":{"append":1}}' | jq . ––– output ––– + [ + { + "total": 10, + "error": "", + "warning": "", + "columns": [ + { + "query": { + "type": "string" + } + } + ], + "data": [ + { + "query": "joe" + }, + { + "query": "jordan" + }, + { + "query": "joseph" + }, + { + "query": "john" + }, + { + "query": "jose" + }, + { + "query": "joy" + }, + { + "query": "joyce" + }, + { + "query": "jonas" + }, + { + "query": "johnson" + }, + { + "query": "major" + } + ] + } + ] ––– input ––– curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"jo","options":{"prepend":0}}' | jq . ––– output ––– + [ + { + "total": 10, + "error": "", + "warning": "", + "columns": [ + { + "query": { + "type": "string" + } + } + ], + "data": [ + { + "query": "jordan" + }, + { + "query": "joe" + }, + { + "query": "joseph" + }, + { + "query": "john" + }, + { + "query": "jose" + }, + { + "query": "joy" + }, + { + "query": "joyce" + }, + { + "query": "jonas" + }, + { + "query": "johnson" + }, + { + "query": "joshua" + } + ] + } + ] ––– input ––– curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"jo","options":{"prepend":1,"append":1}}' | jq . ––– output ––– + [ + { + "total": 10, + "error": "", + "warning": "", + "columns": [ + { + "query": { + "type": "string" + } + } + ], + "data": [ + { + "query": "joe" + }, + { + "query": "jordan" + }, + { + "query": "joseph" + }, + { + "query": "john" + }, + { + "query": "jose" + }, + { + "query": "joy" + }, + { + "query": "joyce" + }, + { + "query": "jonas" + }, + { + "query": "johnson" + }, + { + "query": "major" + } + ] + } + ] ––– input ––– curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"jo","options":{"fuzziness":1,"layouts":"","append":1,"prepend":1}}' | jq . ––– output ––– + [ + { + "total": 10, + "error": "", + "warning": "", + "columns": [ + { + "query": { + "type": "string" + } + } + ], + "data": [ + { + "query": "joe" + }, + { + "query": "jordan" + }, + { + "query": "joseph" + }, + { + "query": "john" + }, + { + "query": "jose" + }, + { + "query": "joy" + }, + { + "query": "joyce" + }, + { + "query": "jonas" + }, + { + "query": "johnson" + }, + { + "query": "major" + } + ] + } + ] ––– input ––– curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"jo","options":{"fuzziness":2,"layouts":"us,ru","append":1,"prepend":1,"expansion_len":2}}' | jq . ––– output ––– + [ + { + "total": 4, + "error": "", + "warning": "", + "columns": [ + { + "query": { + "type": "string" + } + } + ], + "data": [ + { + "query": "joe" + }, + { + "query": "john" + }, + { + "query": "jose" + }, + { + "query": "joy" + } + ] + } + ] ––– input ––– curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"gr","options":{"fuzziness":1,"layouts":"ru","append":1,"prepend":1}}' | jq . ––– output ––– + [ + { + "total": 10, + "error": "", + "warning": "", + "columns": [ + { + "query": { + "type": "string" + } + } + ], + "data": [ + { + "query": "gregory" + }, + { + "query": "grover" + }, + { + "query": "grant" + }, + { + "query": "grady" + }, + { + "query": "gracia" + }, + { + "query": "gregg" + }, + { + "query": "grace" + }, + { + "query": "graham" + }, + { + "query": "griffin" + }, + { + "query": "montenegro" + } + ] + } + ] ––– input ––– curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"jo","options":{"fuzziness":1}}' | jq . ––– output ––– - + [ + { + "total": 10, + "error": "", + "warning": "", + "columns": [ + { + "query": { + "type": "string" + } + } + ], + "data": [ + { + "query": "joe" + }, + { + "query": "jordan" + }, + { + "query": "joseph" + }, + { + "query": "john" + }, + { + "query": "jose" + }, + { + "query": "joy" + }, + { + "query": "joyce" + }, + { + "query": "jonas" + }, + { + "query": "johnson" + }, + { + "query": "major" + } + ] + } + ] + ```
github-actions[bot] commented 1 month ago

clt

👎 CLT tests in test/clt-tests/buddy/ ✅ OK: 3 ❌ Failed: 1 ⏳ Duration: 72s 👉 Check Action Results for commit 474f6a7 Failed tests:

test/clt-tests/buddy/test-fuzzy-autocomplete-http.rec ```diff ––– input ––– rm -f /var/log/manticore/searchd.log; searchd --stopwait > /dev/null; searchd; if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore/searchd.log); then echo 'Buddy started!'; else echo 'Timeout or failed!'; cat /var/log/manticore/searchd.log;fi ––– output ––– Manticore %{SEMVER} %{COMMITDATE}#!/(\sdev)?\s/!#(columnar %{SEMVER} %{COMMITDATE}) (secondary %{SEMVER} %{COMMITDATE}) (knn %{SEMVER} %{COMMITDATE}) Copyright (c) 2001-2016, Andrew Aksyonoff Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com) Copyright (c) 2017-%{YEAR}, Manticore Software LTD (https://manticoresearch.com) [#!/[0-9]{2}:[0-9]{2}.[0-9]{3}/!#] [#!/[0-9]+/!#] using config file '/etc/manticoresearch/manticore.conf' (%{NUMBER} chars)... starting daemon version '%{SEMVER} %{COMMITDATE}#!/(\sdev)?\s/!#(columnar %{SEMVER} %{COMMITDATE}) (secondary %{SEMVER} %{COMMITDATE}) (knn %{SEMVER} %{COMMITDATE})' ... listening on %{IPADDR}:9312 for sphinx and http(s) listening on %{IPADDR}:9306 for mysql listening on %{IPADDR}:9308 for sphinx and http(s) Buddy started! ––– input ––– apt-get update -y > /dev/null; echo $? ––– output ––– 0 ––– input ––– apt-get install jq -y > /dev/null; echo $? ––– output ––– debconf: delaying package configuration, since apt-utils is not installed 0 ––– input ––– php -d memory_limit=-1 ./test/clt-tests/scripts/load_us_names.php 1000 10 1000000 5 2 > /dev/null ––– output ––– ––– input ––– mysql -h0 -P9306 -e "SHOW CREATE TABLE name;" | grep "min_infix_len='2'" | sed "s/.*\(min_infix_len='2'\).*/\1/" ––– output ––– min_infix_len='2' ––– input ––– mysql -h0 -P9306 -e "SELECT COUNT(*) FROM name;" ––– output ––– +----------+ | count(*) | +----------+ | 1000000 | +----------+ ––– input ––– curl -sX POST http://localhost:9308/search -d '{"index":"name","query":{"bool":{"must":[{"match":{"*":"RICH"}}]}},"options":{"fuzzy":true}}' | jq . ––– output ––– + { + "took": 4, + "timed_out": false, + "hits": { + "total": 8380, + "total_relation": "eq", + "hits": [ + { + "_id": 832342, + "_score": 1595, + "_source": { + "username": "RICO RICE" + } + }, + { + "_id": 768099, + "_score": 1562, + "_source": { + "username": "BRICE RICH" + } + }, + { + "_id": 989140, + "_score": 1549, + "_source": { + "username": "LEVI RICO" + } + }, + { + "_id": 869271, + "_score": 1549, + "_source": { + "username": "MARQUITA RICO" + } + }, + { + "_id": 932341, + "_score": 1549, + "_source": { + "username": "JERAMY RICO" + } + }, + { + "_id": 989530, + "_score": 1549, + "_source": { + "username": "RICO RAWLINGS" + } + }, + { + "_id": 939666, + "_score": 1549, + "_source": { + "username": "LAURENE RICO" + } + }, + { + "_id": 944740, + "_score": 1549, + "_source": { + "username": "JOSHUA RICO" + } + }, + { + "_id": 979783, + "_score": 1549, + "_source": { + "username": "RICO SWANK" + } + }, + { + "_id": 929967, + "_score": 1549, + "_source": { + "username": "BIANCA RICO" + } + }, + { + "_id": 914988, + "_score": 1549, + "_source": { + "username": "SUSANNE RICO" + } + }, + { + "_id": 911989, + "_score": 1549, + "_source": { + "username": "RICO IVY" + } + }, + { + "_id": 975060, + "_score": 1549, + "_source": { + "username": "RICO BRENNER" + } + }, + { + "_id": 921107, + "_score": 1549, + "_source": { + "username": "KENYETTA RICO" + } + }, + { + "_id": 996127, + "_score": 1549, + "_source": { + "username": "RICO WHITLOCK" + } + }, + { + "_id": 918295, + "_score": 1549, + "_source": { + "username": "MARLEN RICO" + } + }, + { + "_id": 933392, + "_score": 1549, + "_source": { + "username": "AN RICO" + } + }, + { + "_id": 924527, + "_score": 1549, + "_source": { + "username": "RICO BOEHM" + } + }, + { + "_id": 995638, + "_score": 1549, + "_source": { + "username": "RICO JAEGER" + } + }, + { + "_id": 950970, + "_score": 1549, + "_source": { + "username": "RICO TOTH" + } + } + ] + } + } ––– input ––– curl -sX POST http://localhost:9308/search -d '{"index":"name","query":{"bool":{"must":[{"match":{"*":"EBGRNA"}}]}},"options":{"fuzzy":true}}' | jq . ––– output ––– + { + "took": 3, + "timed_out": false, + "hits": { + "total": 4329, + "total_relation": "eq", + "hits": [ + { + "_id": 999307, + "_score": 1602, + "_source": { + "username": "BERNA WEIR" + } + }, + { + "_id": 929492, + "_score": 1602, + "_source": { + "username": "BERNA CHEUNG" + } + }, + { + "_id": 926530, + "_score": 1602, + "_source": { + "username": "BERNA GRAFF" + } + }, + { + "_id": 935711, + "_score": 1602, + "_source": { + "username": "BERNA PUGA" + } + }, + { + "_id": 985405, + "_score": 1602, + "_source": { + "username": "BERNA WOODWORTH" + } + }, + { + "_id": 943151, + "_score": 1602, + "_source": { + "username": "BERNA REICHERT" + } + }, + { + "_id": 916301, + "_score": 1602, + "_source": { + "username": "BERNA LATHAM" + } + }, + { + "_id": 928383, + "_score": 1602, + "_source": { + "username": "BERNA FRANZ" + } + }, + { + "_id": 868777, + "_score": 1602, + "_source": { + "username": "BERNA HAMLIN" + } + }, + { + "_id": 953801, + "_score": 1602, + "_source": { + "username": "BERNA POSTON" + } + }, + { + "_id": 936413, + "_score": 1602, + "_source": { + "username": "BERNA HOOPER" + } + }, + { + "_id": 987047, + "_score": 1602, + "_source": { + "username": "BERNA WITTE" + } + }, + { + "_id": 977094, + "_score": 1602, + "_source": { + "username": "BERNA ISBELL" + } + }, + { + "_id": 817074, + "_score": 1602, + "_source": { + "username": "BERNA VEGA" + } + }, + { + "_id": 799233, + "_score": 1602, + "_source": { + "username": "BERNA MOREY" + } + }, + { + "_id": 905588, + "_score": 1602, + "_source": { + "username": "BERNA GUILLORY" + } + }, + { + "_id": 798723, + "_score": 1602, + "_source": { + "username": "BERNA LOUIE" + } + }, + { + "_id": 788674, + "_score": 1602, + "_source": { + "username": "BERNA GILL" + } + }, + { + "_id": 790830, + "_score": 1602, + "_source": { + "username": "BERNA ABRAHAM" + } + }, + { + "_id": 889071, + "_score": 1602, + "_source": { + "username": "BERNA ELLIS" + } + } + ] + } + } ––– input ––– curl -sX POST http://localhost:9308/search -d '{"index":"name","query":{"bool":{"must":[{"match":{"*":"ANT"}}]}},"options":{"fuzzy":true}}' | jq . ––– output ––– + { + "took": 93, + "timed_out": false, + "hits": { + "total": 249348, + "total_relation": "eq", + "hits": [ + { + "_id": 855401, + "_score": 1538, + "_source": { + "username": "LUCIANA GANT" + } + }, + { + "_id": 796101, + "_score": 1538, + "_source": { + "username": "ANNIE GANT" + } + }, + { + "_id": 768944, + "_score": 1538, + "_source": { + "username": "ANNITA GANT" + } + }, + { + "_id": 773157, + "_score": 1536, + "_source": { + "username": "JOANNA DENT" + } + }, + { + "_id": 779895, + "_score": 1535, + "_source": { + "username": "ANNALISA KENT" + } + }, + { + "_id": 935071, + "_score": 1531, + "_source": { + "username": "DORIAN GANT" + } + }, + { + "_id": 929610, + "_score": 1531, + "_source": { + "username": "TANDRA GANT" + } + }, + { + "_id": 950473, + "_score": 1531, + "_source": { + "username": "MELANY GANT" + } + }, + { + "_id": 976868, + "_score": 1531, + "_source": { + "username": "STEFANIE GANT" + } + }, + { + "_id": 972411, + "_score": 1531, + "_source": { + "username": "JORDAN GANT" + } + }, + { + "_id": 895992, + "_score": 1531, + "_source": { + "username": "ANTONE GANT" + } + }, + { + "_id": 832469, + "_score": 1531, + "_source": { + "username": "STEFANIE GANT" + } + }, + { + "_id": 984473, + "_score": 1530, + "_source": { + "username": "LEONIE GANT" + } + }, + { + "_id": 959771, + "_score": 1530, + "_source": { + "username": "IRMA GANT" + } + }, + { + "_id": 917932, + "_score": 1530, + "_source": { + "username": "LISHA GANT" + } + }, + { + "_id": 970346, + "_score": 1530, + "_source": { + "username": "LORY GANT" + } + }, + { + "_id": 945404, + "_score": 1530, + "_source": { + "username": "EILENE GANT" + } + }, + { + "_id": 940539, + "_score": 1530, + "_source": { + "username": "NADINE GANT" + } + }, + { + "_id": 997179, + "_score": 1530, + "_source": { + "username": "DORSEY GANT" + } + }, + { + "_id": 934538, + "_score": 1530, + "_source": { + "username": "AUTUMN GANT" + } + } + ] + } + } ––– input ––– curl -sX POST http://localhost:9308/search -d '{"index":"name","query":{"bool":{"must":[{"match":{"*":"tony"}}]}},"options":{"fuzzy":true}}' | jq . ––– output ––– + { + "took": 4, + "timed_out": false, + "hits": { + "total": 4252, + "total_relation": "eq", + "hits": [ + { + "_id": 998183, + "_score": 1532, + "_source": { + "username": "TONA CHOATE" + } + }, + { + "_id": 999284, + "_score": 1532, + "_source": { + "username": "TONA EDMONDS" + } + }, + { + "_id": 932916, + "_score": 1532, + "_source": { + "username": "TONA GROGAN" + } + }, + { + "_id": 924415, + "_score": 1532, + "_source": { + "username": "TONA REYNOSO" + } + }, + { + "_id": 918798, + "_score": 1532, + "_source": { + "username": "TONA KOCH" + } + }, + { + "_id": 973217, + "_score": 1532, + "_source": { + "username": "TONA WINTER" + } + }, + { + "_id": 910284, + "_score": 1532, + "_source": { + "username": "TONA PRENTICE" + } + }, + { + "_id": 943295, + "_score": 1532, + "_source": { + "username": "TONA LAUGHLIN" + } + }, + { + "_id": 988528, + "_score": 1532, + "_source": { + "username": "TONA SUTHERLAND" + } + }, + { + "_id": 928754, + "_score": 1532, + "_source": { + "username": "TONA BARRETT" + } + }, + { + "_id": 916837, + "_score": 1532, + "_source": { + "username": "TONA IVORY" + } + }, + { + "_id": 993840, + "_score": 1532, + "_source": { + "username": "TONA MARQUARDT" + } + }, + { + "_id": 943890, + "_score": 1532, + "_source": { + "username": "TONA ASHE" + } + }, + { + "_id": 988892, + "_score": 1532, + "_source": { + "username": "TONA BOX" + } + }, + { + "_id": 943985, + "_score": 1532, + "_source": { + "username": "TONA DEHART" + } + }, + { + "_id": 971179, + "_score": 1532, + "_source": { + "username": "TONA KOZAK" + } + }, + { + "_id": 981673, + "_score": 1532, + "_source": { + "username": "TONA MCNEILL" + } + }, + { + "_id": 865983, + "_score": 1532, + "_source": { + "username": "TONA PETTIGREW" + } + }, + { + "_id": 982220, + "_score": 1532, + "_source": { + "username": "TONA ARNDT" + } + }, + { + "_id": 947393, + "_score": 1532, + "_source": { + "username": "TONA SEELEY" + } + } + ] + } + } ––– input ––– curl -sX POST http://localhost:9308/search -d '{"index":"name","query":{"bool":{"must":[{"match":{"*":"SMITH"}}]}}, "sort":[{"id":"desc"},{"username":"asc"}], "options":{"fuzzy":true}}' | jq . ––– output ––– + { + "took": 5, + "timed_out": false, + "hits": { + "total": 7170, + "total_relation": "eq", + "hits": [ + { + "_id": 999972, + "_score": 1, + "_source": { + "username": "KEITH SNOW" + } + }, + { + "_id": 999928, + "_score": 1, + "_source": { + "username": "BONITA LEONG" + } + }, + { + "_id": 999858, + "_score": 1, + "_source": { + "username": "TONITA GEARY" + } + }, + { + "_id": 999854, + "_score": 1, + "_source": { + "username": "DOROTHA WISEMAN" + } + }, + { + "_id": 999722, + "_score": 1, + "_source": { + "username": "SCOTT OLSON" + } + }, + { + "_id": 999721, + "_score": 1, + "_source": { + "username": "TONITA ESTRADA" + } + }, + { + "_id": 999263, + "_score": 1, + "_source": { + "username": "RUTHANN GREENE" + } + }, + { + "_id": 999249, + "_score": 1, + "_source": { + "username": "MINH LANDERS" + } + }, + { + "_id": 999126, + "_score": 1, + "_source": { + "username": "MALENA RUTHERFORD" + } + }, + { + "_id": 999119, + "_score": 1, + "_source": { + "username": "RUTHANNE CARDOSO" + } + }, + { + "_id": 998892, + "_score": 1, + "_source": { + "username": "IRWIN SOUTHARD" + } + }, + { + "_id": 998690, + "_score": 1, + "_source": { + "username": "MITCHELL SEITZ" + } + }, + { + "_id": 998580, + "_score": 1, + "_source": { + "username": "RAMONITA SWISHER" + } + }, + { + "_id": 998565, + "_score": 1, + "_source": { + "username": "RAMONITA MAYO" + } + }, + { + "_id": 998538, + "_score": 1, + "_source": { + "username": "ELSA SEITZ" + } + }, + { + "_id": 998525, + "_score": 1, + "_source": { + "username": "SCOTT CARRION" + } + }, + { + "_id": 997944, + "_score": 1, + "_source": { + "username": "RUTHE SCHENK" + } + }, + { + "_id": 997814, + "_score": 1, + "_source": { + "username": "MINH GOODWIN" + } + }, + { + "_id": 997587, + "_score": 1, + "_source": { + "username": "RAMONITA GIBBONS" + } + }, + { + "_id": 997557, + "_score": 1, + "_source": { + "username": "ARTIE SOUTH" + } + } + ] + } + } ––– input ––– curl -sX POST http://localhost:9308/search -d '{"index":"name","query":{"bool":{"must":[{"match":{"*":"SMYTH"}}]}}, "sort":[{"id":"desc"},{"username":"asc"}], "options":{"fuzzy":true}}' | jq . ––– output ––– + { + "took": 5, + "timed_out": false, + "hits": { + "total": 9965, + "total_relation": "eq", + "hits": [ + { + "_id": 999972, + "_score": 1, + "_source": { + "username": "KEITH SNOW" + } + }, + { + "_id": 999854, + "_score": 1, + "_source": { + "username": "DOROTHA WISEMAN" + } + }, + { + "_id": 999722, + "_score": 1, + "_source": { + "username": "SCOTT OLSON" + } + }, + { + "_id": 999498, + "_score": 1, + "_source": { + "username": "DORTHY DUCKWORTH" + } + }, + { + "_id": 999263, + "_score": 1, + "_source": { + "username": "RUTHANN GREENE" + } + }, + { + "_id": 999196, + "_score": 1, + "_source": { + "username": "HEATH MCNAMARA" + } + }, + { + "_id": 999126, + "_score": 1, + "_source": { + "username": "MALENA RUTHERFORD" + } + }, + { + "_id": 999119, + "_score": 1, + "_source": { + "username": "RUTHANNE CARDOSO" + } + }, + { + "_id": 999012, + "_score": 1, + "_source": { + "username": "HEATHER HAMMOCK" + } + }, + { + "_id": 998892, + "_score": 1, + "_source": { + "username": "IRWIN SOUTHARD" + } + }, + { + "_id": 998676, + "_score": 1, + "_source": { + "username": "BURT WOODWORTH" + } + }, + { + "_id": 998525, + "_score": 1, + "_source": { + "username": "SCOTT CARRION" + } + }, + { + "_id": 998220, + "_score": 1, + "_source": { + "username": "MORA WENTWORTH" + } + }, + { + "_id": 998083, + "_score": 1, + "_source": { + "username": "EDMUND ASHWORTH" + } + }, + { + "_id": 997999, + "_score": 1, + "_source": { + "username": "EDYTH CARLISLE" + } + }, + { + "_id": 997944, + "_score": 1, + "_source": { + "username": "RUTHE SCHENK" + } + }, + { + "_id": 997557, + "_score": 1, + "_source": { + "username": "ARTIE SOUTH" + } + }, + { + "_id": 997543, + "_score": 1, + "_source": { + "username": "HEATHER CLAUSEN" + } + }, + { + "_id": 997395, + "_score": 1, + "_source": { + "username": "JON ASHWORTH" + } + }, + { + "_id": 997286, + "_score": 1, + "_source": { + "username": "JOI FARNSWORTH" + } + } + ] + } + } ––– input ––– curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"jo","options":{"layouts":"us,ru"}}' | jq . ––– output ––– + [ + { + "total": 10, + "error": "", + "warning": "", + "columns": [ + { + "query": { + "type": "string" + } + } + ], + "data": [ + { + "query": "joe" + }, + { + "query": "jordan" + }, + { + "query": "joseph" + }, + { + "query": "john" + }, + { + "query": "jose" + }, + { + "query": "joy" + }, + { + "query": "joyce" + }, + { + "query": "jonas" + }, + { + "query": "johnson" + }, + { + "query": "major" + } + ] + } + ] ––– input ––– curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"jo","options":{"append":1}}' | jq . ––– output ––– + [ + { + "total": 10, + "error": "", + "warning": "", + "columns": [ + { + "query": { + "type": "string" + } + } + ], + "data": [ + { + "query": "joe" + }, + { + "query": "jordan" + }, + { + "query": "joseph" + }, + { + "query": "john" + }, + { + "query": "jose" + }, + { + "query": "joy" + }, + { + "query": "joyce" + }, + { + "query": "jonas" + }, + { + "query": "johnson" + }, + { + "query": "major" + } + ] + } + ] ––– input ––– curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"jo","options":{"prepend":0}}' | jq . ––– output ––– + [ + { + "total": 10, + "error": "", + "warning": "", + "columns": [ + { + "query": { + "type": "string" + } + } + ], + "data": [ + { + "query": "jordan" + }, + { + "query": "joe" + }, + { + "query": "joseph" + }, + { + "query": "john" + }, + { + "query": "jose" + }, + { + "query": "joy" + }, + { + "query": "joyce" + }, + { + "query": "jonas" + }, + { + "query": "johnson" + }, + { + "query": "joshua" + } + ] + } + ] ––– input ––– curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"jo","options":{"prepend":1,"append":1}}' | jq . ––– output ––– + [ + { + "total": 10, + "error": "", + "warning": "", + "columns": [ + { + "query": { + "type": "string" + } + } + ], + "data": [ + { + "query": "joe" + }, + { + "query": "jordan" + }, + { + "query": "joseph" + }, + { + "query": "john" + }, + { + "query": "jose" + }, + { + "query": "joy" + }, + { + "query": "joyce" + }, + { + "query": "jonas" + }, + { + "query": "johnson" + }, + { + "query": "major" + } + ] + } + ] ––– input ––– curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"jo","options":{"fuzziness":1,"layouts":"","append":1,"prepend":1}}' | jq . ––– output ––– + [ + { + "total": 10, + "error": "", + "warning": "", + "columns": [ + { + "query": { + "type": "string" + } + } + ], + "data": [ + { + "query": "joe" + }, + { + "query": "jordan" + }, + { + "query": "joseph" + }, + { + "query": "john" + }, + { + "query": "jose" + }, + { + "query": "joy" + }, + { + "query": "joyce" + }, + { + "query": "jonas" + }, + { + "query": "johnson" + }, + { + "query": "major" + } + ] + } + ] ––– input ––– curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"jo","options":{"fuzziness":2,"layouts":"us,ru","append":1,"prepend":1,"expansion_len":2}}' | jq . ––– output ––– + [ + { + "total": 4, + "error": "", + "warning": "", + "columns": [ + { + "query": { + "type": "string" + } + } + ], + "data": [ + { + "query": "joe" + }, + { + "query": "john" + }, + { + "query": "jose" + }, + { + "query": "joy" + } + ] + } + ] ––– input ––– curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"gr","options":{"fuzziness":1,"layouts":"ru","append":1,"prepend":1}}' | jq . ––– output ––– + [ + { + "total": 10, + "error": "", + "warning": "", + "columns": [ + { + "query": { + "type": "string" + } + } + ], + "data": [ + { + "query": "gregory" + }, + { + "query": "grover" + }, + { + "query": "grant" + }, + { + "query": "grady" + }, + { + "query": "gracia" + }, + { + "query": "gregg" + }, + { + "query": "grace" + }, + { + "query": "graham" + }, + { + "query": "griffin" + }, + { + "query": "montenegro" + } + ] + } + ] ––– input ––– curl -sX POST http://localhost:9308/autocomplete -d '{"table":"name","query":"jo","options":{"fuzziness":1}}' | jq . ––– output ––– - + [ + { + "total": 10, + "error": "", + "warning": "", + "columns": [ + { + "query": { + "type": "string" + } + } + ], + "data": [ + { + "query": "joe" + }, + { + "query": "jordan" + }, + { + "query": "joseph" + }, + { + "query": "john" + }, + { + "query": "jose" + }, + { + "query": "joy" + }, + { + "query": "joyce" + }, + { + "query": "jonas" + }, + { + "query": "johnson" + }, + { + "query": "major" + } + ] + } + ] + ```