search

lifeart / code2prompt

Code to Prompt generator
https://lifeart.github.io/code2prompt/
MIT License
0 stars 0 forks source link

Using code2prompt open large git repository and generate prompts for LLMs #4

Open LZING opened 4 days ago

LZING commented 4 days ago

Hi, lifeart. Thank you for a very nice application.

I'm running into a problem right now when dealing with large code repositories. When I'm dealing with small code repositories, code2prompt works great. But when I'm dealing with large repositories, I have a token overflow problem when interacting with LLM.

So how should we deal with large code repositories? Sending only part of the source code will affect the context. Now it seems that only Gemini 1.5pro can handle about 200m tokens, which is the upper limit.

Can you perform tuning on a large code repository? Or do you have any good suggestions?

lifeart commented 4 days ago

Hi @LZING, thank you for question! In this case you need to download repo as zip file and use “import from file” functionality (left button)

lifeart commented 4 days ago

If you need to fit into llm token window - consider selecting only needed file extensions and folders to exclude, it may save a lot of tokens!