search

context-labs / autodoc

Experimental toolkit for auto-generating codebase documentation using LLMs
MIT License
1.93k stars 113 forks source link

incorrect links in references #12

Open andrewhong5297 opened 1 year ago

andrewhong5297 commented 1 year ago

I've noticed sometimes the reference links given are wrong for the source page, not sure how to address this :thinking:

It's pulling the right resource, but all of these should link to that first link given. When I go to the generated markdown (see here https://github.com/duneanalytics/docs/blob/rework/.autodoc/docs/markdown/docs/api/FAQ/other.md), I don't see "All Ethereum and SQL Basics" referenced. So I don't know how it pulled it in as a link somehow.

Maybe the prompt "Always include a list of reference links to GitHub from the context. Links should ONLY come from the context." should be adjusted somehow? I can't follow how context is injected into the createChatChain prompt, maybe the context is mixing up content and source.

image

samheutmaker commented 1 year ago

I fought this issue when building TolyGPT. I'm really not sure what causes it but I have a theory that it is something with how langchain does context injection as this doesn't happen in my projects that do not use langchain.

I'll try to do some digging this week to see if I can come up with a solution, will post my findings here.

andrewhong5297 commented 1 year ago

got it - okay I'll read more into it too.

andrewhong5297 commented 1 year ago

I wonder if using one of these QA with source chains would provide better results?