StampyAI / stampy-chat

Conversational chatbot to answer questions about AI Safety & Alignment based on information retrieved from the Alignment Research Dataset
https://chat.stampy.ai
MIT License
12 stars 5 forks source link

Test new ARD embeddings #84

Closed ccstan99 closed 10 months ago

ccstan99 commented 10 months ago

I was testing the prompteng workflow, which was working nicely with the default setup, but ran into 2 problems depending trying to change the pinecone used. Looks like get_blocks.py returns something leading to KeyError: 'citations'

  1. When switching out the default pinecone from railway to chat.stampy.ai: Added port:8443 worked!

  2. When updating the pinecone key, env, and index to the new values with ARD embeddings: replace() argument 2 must be str, not None Looks like there's some issues with the metadata format in pinecone. I'll wait until that's been resolved.

FraserLee commented 10 months ago

Based on your commit history and the .env.example you provided, I would guess your .env file probably has a parameter named "PINECONE_INDEX" instead of "PINECONE_INDEX_NAME". If I'm correct, then what you're seeing is

PINECONE_INDEX_NAME = os.environ.get('PINECONE_INDEX_NAME') # returns None

PINECONE_INDEX = pinecone.Index(index_name=PINECONE_INDEX_NAME)
# sets the internal pinecone parameter variables['index_name'] to None

# ---- site-packages/pinecone/core/client/configuration.py ----
used_value = variables.get(
                variable_name, variable['default_value']) # variable_name = 'index_name'

# and a while later...

url = url.replace("{" + variable_name + "}", used_value) # this is the actual "replace" that's failing

and changing that parameter name in .env (and .env.example) should fix things.

If I'm not right and it's something else, you can probably get it to generate a stack trace by commenting out the top-level try-catch error reporting. Send it over, and I can try to diagnose any other problems. Hope this helps!

ccstan99 commented 10 months ago

That wasn't it. My .env file was correct but thanks for reminding me to fix the .env.example. There's some mismatch between the pinecone metadata before and after updates so just waiting for those to get resolved first. But the basic functionality seems to be there, just needs to be tested.

ccstan99 commented 10 months ago

Replaced by #93