treygrainger / ai-powered-search

The codebase for the book "AI-Powered Search" (Manning Publications, 2024)
https://aipoweredsearch.com
152 stars 37 forks source link

`Listing 12.12 - Rerun A/B test on new test3 model` fails #103

Open alexott opened 11 months ago

alexott commented 11 months ago

The cell 27 fails with following stacktrace:

KeyError                                  Traceback (most recent call last)
Cell In[27], line 5
      2 purchases = {'test1': 0, 'test3': 0}
      3 for _ in range(0, NUM_USERS):
----> 5     model_name, purchase_made = a_or_b_model(query='transformers dvd', 
      6                                              a_model='test1',
      7                                              b_model='test3')
      8     if purchase_made:
      9         purchases[model_name]+= 1 

Cell In[17], line 20, in a_or_b_model(query, a_model, b_model)
     17 else:
     18     model_name=b_model
---> 20 purchase_made = live_user_query(query=query, 
     21                                model_name=model_name,
     22                                desired=wants_to_purchase,
     23                                meh=might_purchase)
     24 return (model_name, purchase_made)

Cell In[16], line 15, in live_user_query(query, model_name, desired, meh, desired_prob, meh_prob, uninteresting_prob, quit_per_rank_prob)
      1 def live_user_query(query, model_name,
      2                     desired, meh,
      3                     desired_prob=0.15, 
      4                     meh_prob=0.03, 
      5                     uninteresting_prob=0.01,
      6                     quit_per_rank_prob=0.2):
      7     """Live user for 'query' where purchase probability depends on if 
      8        products upc is in one of three sets.
      9        
   (...)
     13        
     14        """   
---> 15     search_results = search(query, model_name, at=10)
     17     results = pd.DataFrame(search_results).reset_index()
     18     for doc in results.to_dict(orient="records"):

Cell In[11], line 32, in search(query, model_name, at, log)
     29 if log:
     30     print(resp)
---> 32 search_results = resp['response']['docs']
     34 for rank, result in enumerate(search_results):
     35     result['rank'] = rank

KeyError: 'response'
softwaredoug commented 7 months ago

Hey Alex I ran through this notebook, and didn't have this error.... are you still getting this?

treygrainger commented 4 months ago

@dcrouch26 - can you please confirm we're good here after your refactors an close this issue? Thanks!