score query returned an invalid score: NaN for doc: 30

[chencong-hakka]

Hi,I met a problem using 7.1.0 in Python 3.6

elasticsearch.exceptions.TransportError: TransportError(500, 'search_phase_execution_exception', 'function score query returned an invalid score: NaN for doc: 30')

[lior-k]

I understand that this is the error that is returned in the HTTP response.

• Please specify the full error in the ES log, including stack trace.
• please share your query & doc id 30 in your index

[Coder2Programmer]

I met the same problem, and I found that query json was truncated. The length of vector is 1024, and its type is np.float32. The following are related logs:

{"type": "server", "timestamp": "2020-07-16T07:34:21,378Z", "level": "DEBUG", "component": "o.e.a.s.TransportSearchAction", "cluster.name": "docker-cluster", "node.name": "4748e0700884", "message": "[vector_index][0], node[lWZE8mLnQX-RE0Acjj7pDw], [P], s[STARTED], a[id=l2PXUui5RvS-_Yq_uBTlEQ]: Failed to execute [SearchRequest{searchType=QUERY_THEN_FETCH, indices=[vector_index], indicesOptions=IndicesOptions[ignore_unavailable=false, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, allow_aliases_to_multiple_indices=true, forbid_closed_indices=true, ignore_aliases=false, ignore_throttled=true], types=[], routing='null', preference='null', requestCache=null, scroll=null, maxConcurrentShardRequests=0, batchedReduceSize=512, preFilterShardSize=128, allowPartialSearchResults=true, localClusterAlias=null, getOrCreateAbsoluteStartMillis=-1, ccsMinimizeRoundtrips=true, source={\"size\":5,\"query\":{\"function_score\":{\"query\":{\"match_all\":{\"boost\":1.0}},\"functions\":[{\"filter\":{\"match_all\":{\"boost\":1.0}},\"script_score\":{\"script\":{\"source\":\"binary_vector_score\",\"lang\":\"knn\",\"params\":{\"field\":\"embedding_vector\",\"cosine\":true,\"vector\":[0.5133344531059265,0.8981565237045288,0.7226642966270447,0.43649062514305115,0.5817468762397766,0.7959572672843933,0.912297248840332,0.7737535834312439,0.9440011382102966,0.7029653191566467,0.35191744565963745,0.4527096748352051,0.1873820573091507,0.9456775784492493,0.4639967978000641,0.6601182222366333,0.8188974857330322,0.07261735945940018,0.030820753425359726,0.8002818822860718,0.7485396862030029,0.6878407597541809,0.8672916889190674,0.12231598794460297,0.594239354133606,0.4081525206565857,0.9326855540275574,0.719282865524292,0.08931779116392136,0.4399399757385254,0.010701382532715797,0.33111152052879333,0.15025007724761963,0.3340179920196533,0.9384557604789734,0.34251725673675537,0.31806057691574097,0.3837665617465973,0.20067793130874634,0.8925186991691589,0.20133976638317108,0.9248546361923218,0.8114219903945923,0.9846741557121277,0.006142137106508017,0.5577014684677124,0.06296732276678085,0.7252670526504517,0.46399518847465515,0.48522093892097473,0.7901471853256226,0.28718024492263794,0.21813148260116577,0.23591448366641998,0.7415750026702881,0.33806684613227844,0.3033522367477417,0.6527672410011292,0.7634222507476807,0.12474363297224045,0.888816773891449,0.9625622630119324,0.8241085410118103,0.6643974184989929,0.960868239402771,0.7408496141433716,0.6742381453514099,0.9405116438865662,0.7906283140182495,0.1137886568903923,0.24076680839061737,0.32023918628692627,0.15763166546821594,0.6216223239898682,0.7072030901908875,0.3343997001647949,0.31105005741119385,0.013760894536972046,0.383798748254776,0.5499693155288696,0.1316985934972763,0.23061811923980713,0.7414465546607971,0.36437663435935974,0.13240016996860504,0.09259818494319916,0.4709242880344391,0.06792411208152771,0.3032580614089966,0.3562561273574829,0.9876983165740967,0.805099606513977,0.8133509159088135,0.9148113131523132,0.619717001914978,0.9077494144439697,0.7182162404060364,0.4400348663330078,0.24087285995483398,0.706297755241394,0.8867979645729065,0.5167534351348877,0.7662191390991211,0.6964025497436523,0.7409865260124207,0.9425304532051086,0.9784534573554993,0.6500651836395264,0.1939525604248047,0.24128124117851257,0.5931104421615601,0.5212786793708801,0.780091404914856,0.07734560966491699,0.23558484017848969,0.8882958292961121,0.4644109606742859,0.8703378438949585,0.07182097434997559,0.3160308003425598,0.5164199471473694,0.7228916883468628,0.8784406781196594,0.8863702416419983,0.13668562471866608,0.25269320607185364,0.20163683593273163,0.8894926309585571,0.21381069719791412,0.7255780100822449,0.6435054540634155,0.4454653859138489,0.5486590266227722,0.8796049356460571,0.9276021718978882,0.2804052233695984,0.2133730798959732,0.6241793632507324,0.4859524667263031,0.14836494624614716,0.20783478021621704,0.11695130914449692,0.4379536509513855,0.12490234524011612,0.7542444467544556,0.29582449793815613,0.9571364521980286,0.14518731832504272,0.35982033610343933,0.7894458770751953,0.6181634664535522,0.6300994157791138,0.20844502747058868,0.8596030473709106,0.6335479021072388,0.7020769119262695,0.16075928509235382,0.19170309603214264,0.5677537322044373,0.8210932016372681,0.11081768572330475,0.9922527074813843,0.19989967346191406,0.3214099109172821,0.5072676539421082,0.9885414838790894,0.46278271079063416,0.837026834487915,0.625415027141571,0.30879005789756775,0.07457561790943146,0.782330334186554,0.8629553318023682,0.20558932423591614,0.5930874943733215,0.8557995557785034,0.729387640953064,0.9669894576072693,0.7333199977874756,0.8981786966323853,0.6985909342765808,0.7216240167617798,0.01499455701559782,4.087433044333011E-4,0.03627586364746094,0.6066653728485107,0.49357181787490845,0.2784483730792999,0.8317250609397888,0.49895939230918884,0.5179208517074585,0.7660344243049622,0.5353879332542419,0.12782667577266693,0.07545290887355804,0.6308155655860901,0.34895235300064087,0.9847381711006165,0.4732021391391754,0.2831277549266815,0.7904419898986816,0.36078372597694397,0.7217469215393066,0.9385364055633545,0.27732914686203003,0.5153811573982239,0.6231673359870911,0.6401715278625488,0.6030075550079346,0.838053822517395,0.873410165309906,0.7518134713172913,0.7627114653587341,0.1425543576478958,0.7112792730331421,0.2755589783191681,0.9819872379302979,0.9195653200149536,0.984527587890625,0.1256435215473175,0.6017645001411438,0.5465248823165894,0.4131745398044586,0.8046371340751648,0.31155329942703247,0.7933050394058228,0.7259060740470886,0.809734046459198,0.7093641757965088,0.06503462791442871,0.61064612865448,0.2413143664598465,0.7300505638122559,0.6555941104888916,0.9579978585243225,0.8745225667953491,0.11966577172279358,0.08256782591342926,0.35430288314819336,0.5559231638908386,0.5919125080108643,0.7609367966651917,0.9761229157447815,0.5909621119499207,0.2778448164463043,0.18347807228565216,0.8954437375068665,0.5748165845870972,0.10391926765441895,0.9346662759780884,0.5383352637290955,0.6471763253211975,0.348436564207077,0.11861789226531982,0.44958460330963135,0.9361848831176758,0.5725523829460144,0.10881160199642181,0.22040081024169922,0.5499479174613953,0.09684495627880096,0.13262899219989777,0.947422981262207,0.6758528351783752,0.8807366490364075,0.2794095575809479,0.7234371900558472,0.7585301995277405,0.9956875443458557,0.8801063895225525,0.23865723609924316,0.43901896476745605,0.3835987448692322,0.37336835265159607,0.8674418330192566,0.9450377821922302,0.7207485437393188,0.3996310234069824,0.22800543904304504,0.9120925068855286,0.6977666616439819,0.9352427124977112,0.824270486831665,0.5641245245933533,0.8265647292137146,0.7522018551826477,0.09135688096284866,0.15076497197151184,0.11120103299617767,0.04252508282661438,0.1956091672182083,0.6214118003845215,0.5917765498161316,0.3852481245994568,0.779454231262207,0.548297107219696,0.5424450635910034,0.20906639099121094,0.05813834071159363,0.9446345567703247,0.4457506537437439,0.13666865229606628,0.8291635513305664,0.22900620102882385,0.7321750521659851,0.5720124840736389,0.4130655527114868,0.9764413833618164,0.21832236647605896,0.5969199538230896,0.24409808218479156,0.6441423892974854,0.8494151830673218,0.2978714406490326,0.5545288324356079,0.13331520557403564,0.5384410619735718,0.9401136040687561,0.27591168880462646,0.3829173743724823,0.29294538497924805,0.6576845645904541,0.6814711689949036,0.6119866371154785,0.476214736700058,0.7876232266426086,0.7232668399810791,0.2335987538099289,0.35477718710899353,0.013948721811175346,0.4101923704147339,0.4616570472717285,0.3212440311908722,0.15401028096675873,0.4171985983848572,0.03678642213344574,0.8952032923698425,6.990327383391559E-4,0.5147296786308289,0.3002746105194092,0.975635826587677,0.45626047253608704,0.4978207051753998,0.5037169456481934,0.3078937530517578,0.6844767332077026,0.046554334461688995,0.7605079412460327,0.10529955476522446,0.33007562160491943,0.8575919270515442,0.4330860674381256,0.3615068197250366,0.4841068685054779,0.5705726146697998,0.34475910663604736,0.10088661313056946,0.2377050220966339,0.3886832892894745,0.822128415107727,0.5575082898139954,0.18647894263267517,0.7515440583229065,0.19838085770606995,0.3115520477294922,0.3693615198135376,0.7005306482315063,0.2874174416065216,0.4857017695903778,0.030736982822418213,0.5617570281028748,0.41618889570236206,0.6451206207275391,0.6695336103439331,0.976571798324585,0.3528245687484741,0.19595232605934143,0.16792376339435577,0.01996232382953167,0.9949263334274292,0.6765201687812805,0.11487487703561783,0.397260457277298,0.5525335669517517,0.6620944142341614,0.11148141324520111,0.21305860579013824,0.5866008996963501,0.8145999908447266,0.6187513470649719,0.23302538692951202,0.915608823299408,0.44279175996780396,0.7425036430358887,0.5813869833946228,0.5669130086898804,0.07397197186946869,0.8894264698028564,0.10681512951850891,0.49178019165992737,0.6801251769065857,0.6274805068969727,0.7194349765777588,0.09699561446905136,0.4270625710487366,0.29220065474510193,0.41617879271507263,0.03449946641921997,0.17882826924324036,0.2697006165981293,0.6196997761726379,0.40924760699272156,0.16304174065589905,0.6593652367591858,0.32166215777397156,0.4593300521373749,0.2344609498977661,0.6010844111442566,0.02758612483739853,0.8247722387313843,0.9484619498252869,0.8817607760429382,0.55771803855896,0.29561150074005127,0.7508109211921692,0.5862782597541809,0.8790472745895386,0.3731481432914734,0.7741039991378784,0.9925550818443298,0.7157257199287415,0.9162726998329163,0.550823986530304,0.07693649083375931,0.1678990125656128,0.6001575589179993,0.21277497708797455,0.015656432136893272,0.851797878742218,0.07549786567687988,0.5865270495414734,0.6683176159858704,0.7820984721183777,0.47776997089385986,0.38499176502227783,0.5439913868904114,0.686354398727417,0.5884109139442444,0.13999521732330322,0.6045131087303162,0.7442726492881775,0.8530673980712891,0.6864749789237976,0.6512414813041687,0.6801245808601379,0.7241649627685547,0.6330992579460144,0.8893535733222961,0.6486185193061829,0.1951751559972763,0.9125135540962219,0.5719124674797058,0.007983890362083912,0.09055965393781662,0.4564252495765686,0.5148431062698364,0.579902708530426,0.763465940952301,0.12913067638874054,0.90", "cluster.uuid": "5WfcGICnSDKVHbC2DXyeow", "node.id": "lWZE8mLnQX-RE0Acjj7pDw" , 
"stacktrace": ["org.elasticsearch.transport.RemoteTransportException: [4748e0700884][172.17.0.4:9300][indices:data/read/search[phase/query]]",
"Caused by: org.elasticsearch.search.query.QueryPhaseExecutionException: Query Failed [Failed to execute main query]",
"at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:305) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:113) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:335) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:355) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.search.SearchService.lambda$executeQueryPhase$1(SearchService.java:340) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.action.ActionListener.lambda$map$2(ActionListener.java:146) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) [elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:58) [elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:73) [elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) [elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:773) [elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.5.0.jar:7.5.0]",
"at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]",
"at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]",
"at java.lang.Thread.run(Thread.java:830) [?:?]",
"Caused by: org.elasticsearch.ElasticsearchException: function score query returned an invalid score: NaN for doc: 0",
"at org.elasticsearch.common.lucene.search.function.FunctionScoreQuery$FunctionFactorScorer.score(FunctionScoreQuery.java:399) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at org.apache.lucene.search.TopScoreDocCollector$SimpleTopScoreDocCollector$1.collect(TopScoreDocCollector.java:69) ~[lucene-core-8.3.0.jar:8.3.0 2aa586909b911e66e1d8863aa89f173d69f86cd2 - ishan - 2019-10-25 23:10:03]",
"at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreRange(Weight.java:242) ~[lucene-core-8.3.0.jar:8.3.0 2aa586909b911e66e1d8863aa89f173d69f86cd2 - ishan - 2019-10-25 23:10:03]",
"at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:229) ~[lucene-core-8.3.0.jar:8.3.0 2aa586909b911e66e1d8863aa89f173d69f86cd2 - ishan - 2019-10-25 23:10:03]",
"at org.elasticsearch.search.internal.CancellableBulkScorer.score(CancellableBulkScorer.java:56) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39) ~[lucene-core-8.3.0.jar:8.3.0 2aa586909b911e66e1d8863aa89f173d69f86cd2 - ishan - 2019-10-25 23:10:03]",
"at org.elasticsearch.search.internal.ContextIndexSearcher.searchInternal(ContextIndexSearcher.java:192) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:173) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:442) ~[lucene-core-8.3.0.jar:8.3.0 2aa586909b911e66e1d8863aa89f173d69f86cd2 - ishan - 2019-10-25 23:10:03]",
"at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:270) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:113) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:335) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:355) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.search.SearchService.lambda$executeQueryPhase$1(SearchService.java:340) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.action.ActionListener.lambda$map$2(ActionListener.java:146) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:63) [elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.action.ActionRunnable.lambda$supply$0(ActionRunnable.java:58) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.action.ActionRunnable$2.doRun(ActionRunnable.java:73) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:773) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.5.0.jar:7.5.0]",
"at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]",
"at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]",
"at java.lang.Thread.run(Thread.java:830) ~[?:?]"] }

So I want to know if exists a solution of the error, thank you @lior-k

[pramod265]

I think the problem is based on the dense vector values. It seems like your vectors are filled with zeros. Thats why cosineSimilarity are returning NaN values.

[Coder2Programmer]

Thanks.

[ghost]

How did you solve this problem in the end? Is it to filter out the zero vector?

[Coder2Programmer]

How did you solve this problem in the end? Is it to filter out the zero vector?

I found that I used an uncorrect vector encoding.

[ghost]

I also encountered the same problem. My data contains a large number of zero vectors, and a field is set to determine whether the vector is a zero vector. How do I use this flag to filter out the zero vector?

[Coder2Programmer]

I also encountered the same problem. My data contains a large number of zero vectors, and a field is set to determine whether the vector is a zero vector. How do I use this flag to filter out the zero vector?

I used Python3 to encode, and not use dfloat32 = np.dtype('>f4') other than dfloat32 = float. Therefore an error appeared.

[ghost]

I also encountered the same problem. My data contains a large number of zero vectors, and a field is set to determine whether the vector is a zero vector. How do I use this flag to filter out the zero vector?

I used Python3 to encode, and not use dfloat32 = np.dtype('>f4') other than dfloat32 = float. Therefore an error appeared.

The problem I encountered is different from yours, but thank you anyway.

[Kranthi596rguktian]

Any one found the solution for this problem, please post the solution here