context-labs / autodoc

Experimental toolkit for auto-generating codebase documentation using LLMs
MIT License
1.93k stars 113 forks source link

Skip files and folders that are unchanged when reindexing #19

Closed andrewhong5297 closed 1 year ago

andrewhong5297 commented 1 year ago

This PR adds a checksum field to every json output of file/folder processing, and then checks against that when estimating/indexing again. This fixes issue #7

I've tested that it definitely works for files - folders are skipped in estimation, but it is the same logic so I don't see why it wouldn't work.

You can see the verbose results of an estimation below, I already reindexed once and then made more edits to just two files. I didn't change the "Proccessing X files..." or folders text because it uses traverseFileSystem instead of processRepository to create a count.

image