search

treygrainger / ai-powered-search

The codebase for the book "AI-Powered Search" (Manning Publications, 2024)
https://aipoweredsearch.com
152 stars 37 forks source link

`Listing 12.13 - Fully Automated LTR Loop` fails #104

Closed alexott closed 6 months ago

alexott commented 11 months ago

stacktrace:

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[29], line 30
     28 train, test = test_train_split(sdbn, train=0.8) 
     29 ranksvm_ltr(train, model_name='exploit', feature_set=exploit_feature_set)
---> 30 eval_model(test, model_name='exploit', sdbn=new_sdbn) 
     32 # ===============
     33 # EXPLORE
     35 explore_feature_set = [
     36     {
     37       "name" : "manufacturer_match",
   (...)
     66       }
     67     }]

NameError: name 'new_sdbn' is not defined
softwaredoug commented 7 months ago

Hey Alex, with a fresh checkout I haven't had this problem

{!terms f=upc}27242815414,600603132872,827396513927,600603141003,885170033412,883929140855,24543672067,813774010904,36725617605,786936817232,36725608443,719192580374,25192073007,75993997675,36725608894,786936817218,711719983156,22265052211,883929197965,25192107191,22265004517,58231306590,58231300826,23942972389,826663129342,885170038875,786936805017,711719804604,36725608511,27242809710
Searching products [Status: 200]
Missing doc 600603132872
Missing doc 600603141003
{!terms f=upc}600603141003,27242813908,600603132872,36725608894,25192107191,36725608511,605342041546,883929140855,786936817232,22265004517,719192580374,24543742180,36725608443,23942973416,36725617605,27242815414,826663129342,58231306590,27242809758,36725608436,36725617506,711719983156,186156000290,58231300826,23942972389,786936805017,25192073007,885170038875,711719804604,27242809710
Searching products [Status: 200]
Missing doc 600603141003
Missing doc 600603132872
{!terms f=upc}856751002097,48231011396,84691226727,74108007469,12505525766,36725578241,48231011402,12505527456,74108096487,36725561977,84691226703,665331101927,783722274422,14381196320,77283045400,74108056764,883049066905,12505451713,36172950027,883929085118
Searching products [Status: 200]
{!terms f=upc}803238004525,27242799127,709483027855,615104173552,27242740389,878615035287,848447000005,46838046100,673796100317,17817553476,885909412006,723755894396,21331350091,731855020454,46838044106,615104194557,600603124570,50644614705,27242708242,743878021295,848447000135,17817542371,878615035294,27242745100,27242798236,709483027848,878615025226,27242807785,848447000081,27242801585
Searching products [Status: 200]
Missing doc 600603124570
{!terms f=upc}885909457588,885909472376,821793013776,635753493559,722868830062,885909471812,92636260712,886111271283,885909457632,885909393404,885909457601,716829772249,885909457595,600603132827,27242798236,884962753071,886111287055,635753490879,843404073153,610839379408
Searching products [Status: 200]
Missing doc 600603132827
{!terms f=upc}885909510344,600603140631,885909538041,705105070803,660715215252,56035362545,883929153121,884720011757,722868816622,885909538058,885909528912,636926048583,722868787595,600603125065,685387305636,899794006745,600603132827,97855069917,885909528905,885909420445,600603133237,885909420438,843404064434,885909538027,885909459834,885909343874,885909397723,885909394494,885909459858,885909538034
Searching products [Status: 200]
Missing doc 600603140631
Missing doc 600603125065
Missing doc 600603132827
Missing doc 600603133237
{!terms f=upc}814916011896,9781400532711,814916014361,814916011872,814916014590,814916014385,813580017906,814916010202,814916010240,9781400532650,813580018514,814916010288,813580018491,814916010219,92636257521,814916014606,813580015261,814916010233,813580018361,843404077182,813580015247,885909457595,716829772249,813580015933,813580015698,813580015155,813580015223,813580015636,813580015292,813580016107
Searching products [Status: 200]
{!terms f=upc}36725236271,883393003458,36725234789,882777064009,22265004289,27242817197,729507810218,885170042704,812491010310,812491010334,827912072969,74000373105,884483335329,827912068467,827912068474,97278016000,696211503197,605342041546,882777064207,885170042667,13803112610,729507813059,22265004258,719192579996,600603139758,36725235564,22265004302,723755834491
Searching products [Status: 200]
Missing doc 600603139758
{!terms f=upc}885909436705,885909464043,885909431618,885909454396,885909432257,885909437023,885909436002,885909464036,885909463626,885909398577,885909377794,885909282883,885909398584,4712052310841,885909262236,94922876065,13803133929,27242827790,886112432140,650450759814,13803123470,36725729872,36725617421,28287033924,99802970796,27242836228,600603123061,400027951918,73502031827,29521842944
Searching products [Status: 200]
Missing doc 600603123061
{!terms f=upc}600603135088,9781400532711,9781400532650,813580016961,9780594033899,814916014606,9781400532629,9780594033929,716829772249,9781616874452,813580017937,9780594033912,660543005827,814916011896,814916011872,9781400532667,9780594033936,9781616826055,814916010240,600603135101,813580016824,36725234642,716829782446,843404077427,92636257521,813580013571,813580017043,813580017074,813580017067
Searching products [Status: 200]
Missing doc 600603135101
{!terms f=upc}50644555190,97360719840,97361427546,97360719741,13964123296,97360719642,30206696622,97363485049,738572128920,97360717648,97361166247,13964123302,5051368213637,29757201560,46034897179,883929139446,742725280410,635753490541,97361301747,31398121381,27242829619,97360743548,97360719147,791149900183,52824803121,97361311944,12505226021,50694439860,36725578340,36725235564
Searching products [Status: 200]
{!terms f=upc}23272342630,24543742180,24543742098,23272341381,885370332889,886973561621,14633169522,24543781875,708431390614,884116069973,30206742121,24543742074,883929154012,23272342654,24543560067,883929200887,14633169546,24543023920,24543263739,400032015667,886971404722,23272332389,24543559856,827969295021,738572121921,883929094561,827969295120,24543203094,827969422021,97361455044
Searching products [Status: 200]
{!terms f=upc}97360810042,400192926087,97363560449,97363532149,93624956037,47875842328,47875841420,25192107191,786936817218,36725235564,24543701538,47875841369,47875841406,24543750949,47875842335
Searching products [Status: 200]
{!terms f=upc}97360810042,400192926087,97363560449,97363532149,93624956037,47875842328,47875841420,25192107191,786936817218,24543701538,47875841369,36725235564,47875841406,24543750949,47875842335
Searching products [Status: 200]
{!terms f=upc}97360810042,400192926087,97363560449,97363532149,93624956037,47875842328,47875841420,25192107191,786936817218,47875841369,36725235564,24543701538,47875841406,24543750949,47875842335
Searching products [Status: 200]
{!terms f=upc}97360810042,400192926087,97363560449,97363532149,93624956037,47875842328,47875841420,25192107191,786936817218,36725235564,47875841369,24543701538,47875841406,24543750949,47875842335
Searching products [Status: 200]
{!terms f=upc}803238004525,27242799127,709483027855,615104173552,27242740389,878615035287,848447000005,46838046100,17817553476,673796100317,885909412006,723755894396,21331350091,731855020454,46838044106,615104194557,600603124570,50644614705,743878021295,27242708242,848447000135,17817542371,878615035294,27242745100,27242798236,878615025226,709483027848,27242807785,848447000081,27242801585
Searching products [Status: 200]
Missing doc 600603124570
{!terms f=upc}36725236271,883393003458,36725234789,882777064009,22265004289,27242817197,729507810218,812491010310,885170042704,827912072969,74000373105,812491010334,884483335329,827912068467,827912068474,97278016000,696211503197,605342041546,882777064207,885170042667,13803112610,729507813059,22265004258,719192579996,600603139758,36725235564,22265004302,723755834491
Searching products [Status: 200]
Missing doc 600603139758
{!terms f=upc}36725236271,883393003458,36725234789,882777064009,22265004289,27242817197,729507810218,812491010310,885170042704,827912072969,812491010334,74000373105,884483335329,827912068467,97278016000,827912068474,696211503197,605342041546,882777064207,885170042667,13803112610,729507813059,22265004258,719192579996,600603139758,36725235564,22265004302,723755834491
Searching products [Status: 200]
Missing doc 600603139758
{!terms f=upc}885909436705,885909464043,885909431618,885909454396,885909432257,885909437023,885909464036,885909436002,885909398577,885909463626,885909377794,885909282883,885909398584,885909262236,4712052310841,94922876065,13803133929,27242827790,886112432140,650450759814,13803123470,36725729872,36725617421,28287033924,99802970796,27242836228,600603123061,400027951918,73502031827,29521842944
Searching products [Status: 200]
Missing doc 600603123061
{!terms f=upc}885909510344,600603140631,885909538041,705105070803,660715215252,56035362545,883929153121,884720011757,722868816622,885909538058,885909528912,636926048583,722868787595,600603125065,685387305636,899794006745,600603132827,97855069917,885909528905,885909420445,885909420438,600603133237,885909538027,843404064434,885909459834,885909343874,885909397723,885909394494,885909459858,885909538034
Searching products [Status: 200]
Missing doc 600603140631
Missing doc 600603125065
Missing doc 600603132827
Missing doc 600603133237
{!terms f=upc}814916011896,9781400532711,814916014361,814916014590,814916011872,814916014385,813580017906,814916010240,814916010202,813580018514,9781400532650,814916010288,813580018491,814916010219,92636257521,814916014606,813580015261,814916010233,843404077182,813580018361,813580015247,716829772249,885909457595,813580015698,813580015933,813580015223,813580015155,813580015636,813580015292,813580016107
Searching products [Status: 200]
{!terms f=upc}814916011896,9781400532711,814916014361,814916011872,814916014590,814916014385,813580017906,814916010202,814916010240,9781400532650,813580018514,813580018491,814916010288,814916010219,92636257521,814916014606,813580015261,814916010233,813580018361,843404077182,813580015247,716829772249,885909457595,813580015933,813580015223,813580015155,813580015698,813580015636,813580015292,813580016107
Searching products [Status: 200]
{!terms f=upc}27242815414,600603132872,827396513927,600603141003,885170033412,883929140855,24543672067,813774010904,786936817232,36725617605,719192580374,36725608443,25192073007,75993997675,36725608894,786936817218,711719983156,22265052211,883929197965,25192107191,22265004517,58231306590,58231300826,23942972389,826663129342,885170038875,786936805017,711719804604,36725608511,27242809710
Searching products [Status: 200]
Missing doc 600603132872
Missing doc 600603141003
{!terms f=upc}97363560449,97361312804,97361312743,97363455349,97361372389,97363532149,879862003517,93624995012,47875842328,708056579746,47875332911,47875819733,708056579739,879862003524,93624974918,47875839090
Searching products [Status: 200]
*:*
{'responseHeader': {'zkConnected': True, 'status': 0, 'QTime': 2, 'params': {'json': '{"fields": ["upc", "name", "manufacturer", "score"], "limit": 1, "params": {"q": "*:*", "sort": "random_0.1856832131476962 DESC"}}'}}, 'response': {'numFound': 48204, 'start': 0, 'maxScore': 1.0, 'numFoundExact': True, 'docs': [{'upc': '49925004021', 'name': 'Reasonable Doubt [PA] - CD', 'manufacturer': 'Roc-A-Fella Catalog (EMD)', 'score': 1.0}]}}
softwaredoug commented 7 months ago

BTW I started the notebooks with

docker-compose up -d --no-deps --build

After a fresh checkout (I have a new laptop compared to last time I used this repo)

alexott commented 6 months ago

@softwaredoug It's failing with NO such queryParser 'ltr' - see #120 - maybe it's related to recent Solr update? I rebuilt all containers from scratch

treygrainger commented 6 months ago

Yeah @alexott @softwaredoug It's I reported this issue on the PR, as well. https://github.com/treygrainger/ai-powered-search/pull/114

alexott commented 6 months ago

@softwaredoug check that when your docker-compose starts containers it doesn't hanve messages like this:

Error: error creating container storage: the container name "aips-solr" is already in use by "237819dbbdbbda29022d58196337ba2c312d923c5280706bbbdcb00d0cd346d6". You have to remove that container to be able to reuse that name.: that name is already in use
exit code: 125

then it's related to #131

alexott commented 6 months ago

Fixed in the latest versions