search

gusye1234 / nano-graphrag

A simple, easy-to-hack GraphRAG implementation
MIT License
1.62k stars 157 forks source link

robust json parsing & entity extraction progress log #55

Closed rangehow closed 1 month ago

rangehow commented 2 months ago
  1. The JSON generated by large models may have issues such as being incomplete, redundant, or having formatting errors. The original solution was too brief and provided few prompts, so we made some improvements to address these.
  2. Entity extraction is the most time-consuming part of this project, but there was no explicit progress indicator. We added a percentage indicator to let users know the current progress more clearly.
codecov[bot] commented 2 months ago

Codecov Report

Attention: Patch coverage is 90.74074% with 5 lines in your changes missing coverage. Please review.

Project coverage is 94.25%. Comparing base (1a85851) to head (ba3b2be). Report is 13 commits behind head on main.

Files with missing lines Patch % Lines
nano_graphrag/_utils.py 90.56% 5 Missing :warning:
Additional details and impacted files ```diff @@ Coverage Diff @@ ## main #55 +/- ## ======================================= Coverage 94.25% 94.25% ======================================= Files 12 12 Lines 1288 1288 ======================================= Hits 1214 1214 Misses 74 74 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

NazimHAli commented 1 month ago

I didn't see this PR until now, yesterday I ended up adding tqdm to every task so that all the progress bars were available by default. Happy to help with this if needed :+1:

rangehow commented 1 month ago

I didn't see this PR until now, yesterday I ended up adding tqdm to every task so that all the progress bars were available by default. Happy to help with this if needed.

Hi @NazimHAli. You can bring up a new PR, and I will reorganize the submitted code by removing the percentage related parts. But it's better to do it quickly, because tonight this PR will be merged.

rangehow commented 1 month ago

@gusye1234 cc

NazimHAli commented 1 month ago

I didn't see this PR until now, yesterday I ended up adding tqdm to every task so that all the progress bars were available by default. Happy to help with this if needed.

Hi @NazimHAli. You can bring up a new PR, and I will reorganize the submitted code by removing the percentage related parts. But it's better to do it quickly, because tonight this PR will be merged.

I won't have time to do it before this gets merged. You can continue with this change and I can do a follow-up later.