Closed valaises closed 4 months ago
vecdb indexing is stuck forever when indexing a project with some python packages in it, node_modules, etc. Guess, vecdb search quality won't be great either when some many garbage in it
there's existing filtering pipeline in python, could be useful https://github.com/smallcloudai/data-collection/blob/main/github/scripts/stage4_diffs/filtration/plain/prefilter.py
vecdb indexing is stuck forever when indexing a project with some python packages in it, node_modules, etc. Guess, vecdb search quality won't be great either when some many garbage in it