Seperate Issue Indexing

geprog / CodeCaptain

Ask codecaptain everything you want to know about the source-code or issues of your project

https://codecaptain.ai

GNU Affero General Public License v3.0

7 stars 3 forks source link

Seperate Issue Indexing #25

Closed aaa006bd closed 5 months ago

aaa006bd commented 1 year ago

@anbraten During my testing with bigger repositories I have encountered an error related to max token( Probably because we are trying to tokenize a l whole lot of texts from issues) which leads to faulty import. We should try to separate this. Maybe we can have a button that would only reindex the issues related to the repo when we want.

anbraten commented 1 year ago

Those are two issues AFAIK:

for the max tokens we can use a splitter or one of the other langchain summarzie etc helpers
for the issues, we should definitely still include them (as they provide quite some helpful context), but we need to improve the fetching to only get updated issues after the first complete fetch when pressing sync repo

aaa006bd commented 1 year ago

@anbraten All right I will then look up in the langchain for proper size tokenizations. I am opening a PR for this in this issue.